Feedback-ID: i131946ab:Fastmail
Message-ID: <c2758429-b404-5feb-6ac4-2ced0b3ad755@enterprisedb.com>
Date: Thu, 16 Mar 2023 11:26:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.8.0
Subject: Re: Transparent column encryption
Content-Language: en-US
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
References: <d9dd3634-6154-b0b7-f3a9-0d86dad7b8ff@enterprisedb.com>
 <bdb6433c-2d4e-8ae3-3bda-913f0b53fbb8@enterprisedb.com>
 <CALDaNm393nCjM1GenCem=9kOfZxbSadsTSn48WDVomwPnfnCFw@mail.gmail.com>
 <1034b6f1-ccab-8a22-c843-71104ebedf01@enterprisedb.com>
 <40c43d0d-ae4b-fe34-2667-771b3718384e@enterprisedb.com>
 <47e17f03-6595-6c09-8978-7a1581cad6f2@enterprisedb.com>
 <7a430d89-233b-5cd7-6892-9fed06695d39@enterprisedb.com>
 <20230312001125.ijeu4imbwlhj5flf@awork3.anarazel.de>
 <43a88695-5e0b-4379-94cf-0392fdea4be9@enterprisedb.com>
 <20230313204119.4mkepdvixcxrwpsc@awork3.anarazel.de>
 <20230313211108.zge6vnqukexkih62@awork3.anarazel.de>
From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
In-Reply-To: <20230313211108.zge6vnqukexkih62@awork3.anarazel.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/c2758429-b404-5feb-6ac4-2ced0b3ad755%40enterprisedb.com>
Precedence: bulk

On 13.03.23 22:11, Andres Freund wrote:
>> It adds branches, and it makes tupledescs wider. In tight spots, such as
>> printtup, that can hurt, even if the branches aren't ever entered.
> In fact, I do see a noticable, but not huge, regression:

I tried to reproduce your measurements, but I don't have the CPU 
affinity tools like that on macOS, so the differences were lost in the 
noise.  (The absolute numbers looked very similar to yours.)

I can reproduce a regression if I keep adding more columns to 
pg_attribute, like 8 OID columns does start to show.

I investigated whether I could move some columns to the 
"variable-length" part outside the tuple descriptor, but that would 
require major surgery in heap.c and tablecmds.c (MergeAttributes() ... 
shudder), and also elsewhere, where you currently only have a tuple 
descriptor for looking up stuff.

How do we proceed here?  A lot of user-facing table management stuff 
like compression, statistics, collation, dropped columns, and probably 
future features like column reordering or periods, have to be 
represented in pg_attribute somehow, at least in the current 
architecture.  Do we hope that hardware keeps up with the pace at which 
we add new features?  Do we need to decouple tuple descriptors from 
pg_attribute altogether?  How do we decide what goes into the tuple 
descriptor and what does not?  I'm interested in addressing this, 
because obviously we do want the ability to add more features in the 
future, but I don't know what the direction should be.