Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1peuL6-0004PW-59 for pgsql-hackers@arkaria.postgresql.org; Wed, 22 Mar 2023 09:01:00 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1peuL3-0005ZC-Ch for pgsql-hackers@arkaria.postgresql.org; Wed, 22 Mar 2023 09:00:57 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1peuL2-0005Z1-Fa for pgsql-hackers@lists.postgresql.org; Wed, 22 Mar 2023 09:00:57 +0000 Received: from wnew1-smtp.messagingengine.com ([64.147.123.26]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1peuKy-00022k-9N for pgsql-hackers@postgresql.org; Wed, 22 Mar 2023 09:00:56 +0000 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.west.internal (Postfix) with ESMTP id AA7132B06A4A; Wed, 22 Mar 2023 05:00:48 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 22 Mar 2023 05:00:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1679475648; x=1679482848; bh=v9tfKuyOGU6eJupY0G6o5tV2fNFUHXbkSwm 9AUSkb/0=; b=sZGwnisIFGdQFg4lZa1zvTK8lv1x/zCOlO+EwpYEdN+cUjpN1/j u+ctPgMzlmFoE1I2ARTOhtQRgT0ME0mOabuBR3yQDRtsa+PIBQ3YzbPZq+2hOQoB /XC9/w0oQ4c8WAMzNXPf+21971u7z1WwVmzQoEfL5MZ9h8x0JBMPaJDJ+O1/I6Ja 4svHrRjuv22hFHIHGzG5tPwqLzzHwqPH+V2HgwLB7hPCMTp/pO+lX9Qpm/3oZeTF e6XEldf4/UMmeaOXygatP3SxtbMThOO+Iynmrlb/UGRDM59PTVDnbB2XM0ZopQ6c kN2TTG+HWZSmirXeOdbr0/Zr9Q5WvIy2BqA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdeguddgudeftdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomheprfgv thgvrhcugfhishgvnhhtrhgruhhtuceophgvthgvrhdrvghishgvnhhtrhgruhhtsegvnh htvghrphhrihhsvggusgdrtghomheqnecuggftrfgrthhtvghrnhepheelffeggedujeet iefhleetuddvieffhfffvdejvdffgeejkeduleduheduieefnecuvehluhhsthgvrhfuih iivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphgvthgvrhdrvghishgvnhhtrhgr uhhtsegvnhhtvghrphhrihhsvggusgdrtghomh X-ME-Proxy: Feedback-ID: i131946ab:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 22 Mar 2023 05:00:47 -0400 (EDT) Message-ID: <62cbdf73-e933-3020-18d2-9b854159032f@enterprisedb.com> Date: Wed, 22 Mar 2023 10:00:45 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Subject: Re: Transparent column encryption Content-Language: en-US To: Andres Freund Cc: pgsql-hackers References: <40c43d0d-ae4b-fe34-2667-771b3718384e@enterprisedb.com> <47e17f03-6595-6c09-8978-7a1581cad6f2@enterprisedb.com> <7a430d89-233b-5cd7-6892-9fed06695d39@enterprisedb.com> <20230312001125.ijeu4imbwlhj5flf@awork3.anarazel.de> <43a88695-5e0b-4379-94cf-0392fdea4be9@enterprisedb.com> <20230313204119.4mkepdvixcxrwpsc@awork3.anarazel.de> <20230313211108.zge6vnqukexkih62@awork3.anarazel.de> <20230316163648.q3wgjinmad6225t3@awork3.anarazel.de> <20230321174758.yfx2ulnqekgtnenv@awork3.anarazel.de> From: Peter Eisentraut In-Reply-To: <20230321174758.yfx2ulnqekgtnenv@awork3.anarazel.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 21.03.23 18:47, Andres Freund wrote: > On 2023-03-21 18:05:15 +0100, Peter Eisentraut wrote: >> On 16.03.23 17:36, Andres Freund wrote: >>> Maybe a daft question, but why do we need a separate type and typmod for >>> encrypted columns? Why isn't the fact that the column is encrypted exactly one >>> new field, and we use the existing type/typmod fields? >> >> The way this is implemented is that for an encrypted column, the real >> atttypid and atttypmod are one of the encrypted special types >> (pg_encrypted_*). That way, most of the system doesn't need to care about >> the details of encryption or whatnot, it just unpacks tuples etc. by looking >> at atttypid, atttyplen, etc., and queries on encrypted data behave normally >> by just looking at what operators etc. those types have. This approach >> heavily contains the number of places that need to know about this feature >> at all. > > I get that for the type, but why do we need the typmod duplicated as well? Earlier patch versions didn't do that, but that got really confusing about which type the typmod really belonged to, since code currently assumes that typid+typmod makes sense. Earlier patch versions had three fields (usertypid, keyid, encalg), and then I changed it to (usertypid, usertypmod, keyid) and instead placed the encalg into the real typmod, which made everything much cleaner.