Date: Sat, 21 Oct 2023 21:56:13 -0400
From: Bruce Momjian <bruce@momjian.us>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Laurenz Albe <laurenz.albe@cybertec.at>, suchithjn22@gmail.com,
	PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: The documentation for storage type 'plain' actually allows
 single byte header
Message-ID: <ZTSBPankJQ9_kfXF@momjian.us>
References: 
 <167336599095.2667301.15497893107226841625@wrigleys.postgresql.org>
 <d99b3e0beebdde0741d28035aaea7f4af56fa050.camel@cybertec.at>
 <ZRdNjrI6lUQbD90Y@momjian.us>
 <4977c12e127250fe9777ae57f6f5884b8f5c84ae.camel@cybertec.at>
 <1421365.1696027552@sss.pgh.pa.us>
 <ZTMt1XxPmZekFXBL@momjian.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZTMt1XxPmZekFXBL@momjian.us>
Archived-At: 
 <https://www.postgresql.org/message-id/ZTSBPankJQ9_kfXF%40momjian.us>
Precedence: bulk

On Fri, Oct 20, 2023 at 09:48:05PM -0400, Bruce Momjian wrote:
> Here is the original thread from pgsql-docs:
> 
> 	https://www.postgresql.org/message-id/flat/167336599095.2667301.15497893107226841625%40wrigleys.postgresql.org
> 
> The report is about single-byte headers being used for varlena values
> with PLAIN storage.
> 
> Here is the reproducible report:
> 
> 	CREATE EXTENSION pageinspect;
> 	CREATE TABLE test(a VARCHAR(10000) STORAGE PLAIN);
> 	INSERT INTO test VALUES (repeat('A',10));
> 	
> 	Now peek into the page with pageinspect functions
> 	
> 	SELECT left(encode(t_data, 'hex'), 40) FROM
> 	heap_page_items(get_raw_page('test', 0));
> 	
> 	This returned value of "1741414141414141414141".
> 	Here the first byte 0x17 = 0001 0111 in binary.
> 	Length + 1 is stored in the length bits (1-7). So Len = 0001011-1 = (11-1)
> 	[base-10] = 10 [base-10]
> 	which exactly matches the expected length. Further the data "41" repeated 10
> 	times also indicates character A (65 or 0x41 in ASCII) repeated 10 times.
> 
> I researched this and thought it would be a case where we were lacking a
> check before creating a single-byte header, but I couldn't find anything
> missing.  I think the problem is that the _source_ tupleDesc attstorage
> attribute is being used to decide if we should use a short header, while
> it is really the storage type of the destination that we should be
> checking.  Unfortunately, I don't think the destination is accessible at
> the location were we are deciding about a short header.
> 
> I am confused how to proceed.  I feel we need to fully understand why
> this happening before we adjust anything.  Here is a backtrace --- the
> short header is being created in fill_val() and the attstorage value
> there is 'x'/EXTENDED.

I did some more research.  It turns out that the source slot/planSlot is
populating its pg_attribute information via makeTargetEntry() and it
has no concept of a storage type.

Digging further, I found that we cannot get rid of the the use of
att->attstorage != TYPSTORAGE_PLAIN in macros ATT_IS_PACKABLE and
VARLENA_ATT_IS_PACKABLE macros in src/backend/access/common/heaptuple.c
because there are internal uses of fill_val() that can't handle packed
varlena headers.

I ended up with a doc patch that adds a C comment about this odd
behavior and removes doc text about PLAIN storage not using packed
headers.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.