agora inbox for pgsql-bugs@postgresql.org
help / color / mirror / Atom feedBUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row
3+ messages / 2 participants
[nested] [flat]
* BUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row
@ 2026-08-21 09:23 PG Bug reporting form <noreply@postgresql.org>
2026-08-21 14:37 ` Re: BUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row Andrey Rachitskiy <pl0h0yp1@gmail.com>
0 siblings, 1 reply; 3+ messages in thread
From: PG Bug reporting form @ 2026-08-21 09:23 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: 17801022106@163.com
The following bug has been logged on the website:
Bug reference: 19636
Logged by: anliuan
Email address: 17801022106@163.com
PostgreSQL version: 19beta1
Operating system: centos
Description:
When a PL/pgSQL function/procedure inserts an array value that arrives as an
expanded object (via EOH_flatten_into), the resulting tuple stores the
varlena with a 4-byte header instead of converting it to a
1-byte short varlena header, wasting 3 bytes per row for arrays <= 127
bytes total size.
Plain INSERT correctly uses the short varlena format; only the PL/pgSQL
expanded-object path is affected.
Steps to reproduce:
CREATE TABLE tt2(a text[]);
INSERT INTO tt2 VALUES('{aaaaa}'); -- plain INSERT
CREATE OR REPLACE FUNCTION call_insert_acl(racl text[]) RETURNS int AS $$
BEGIN
INSERT INTO tt2 VALUES(racl);
RETURN 1;
END;
$$ LANGUAGE plpgsql;
SELECT call_insert_acl('{aaaaa}'); -- PL/pgSQL INSERT
SELECT a, pg_column_size(a) FROM tt2;
Expected result: Both rows should have the same pg_column_size (33 bytes —
short varlena with 1-byte header).
Actual result:
┌─────────────────┬────────────────┬───────────────────────────────┐
│ row │ pg_column_size │ header format │
├─────────────────┼────────────────┼───────────────────────────────┤
│ plain INSERT │ 33 │ short varlena (1-byte header) │
├─────────────────┼────────────────┼───────────────────────────────┤
│ PL/pgSQL INSERT │ 36 │ 4-byte header │
└─────────────────┴────────────────┴───────────────────────────────┘
Root cause:
In src/backend/access/common/heaptuple.c, heap_fill_tuple has four varlena
paths. The EOH_flatten_into path (for expanded objects) produces a
4-byte-header varlena and writes it directly, without checking
VARATT_CAN_MAKE_SHORT:
/* Current code — line ~229 */
if (VARATT_IS_EXTERNAL_EXPANDED(val))
{
ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
data = (char *) ATT_ALIGN_NOMINAL(data, att[i]->attalign);
data_length = EOH_get_flat_size(eoh);
EOH_flatten_into(eoh, data, data_length);
/* BUG: no VARATT_CAN_MAKE_SHORT check here */
}
The normal 4-byte-header path correctly performs this check:
/* Normal path — line ~240 */
else if (VARLENA_ATT_IS_PACKABLE(att[i]) && VARATT_CAN_MAKE_SHORT(val))
{
data_length = VARATT_CONVERTED_SHORT_SIZE(val);
SET_VARSIZE_SHORT(data, data_length);
}
heap_compute_data_size has the same omission — it uses EOH_get_flat_size()
without considering the short-header size reduction, so the size calculation
and fill logic are consistently wrong (no memory
overrun, just wasted space).
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: BUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row
2026-08-21 09:23 BUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row PG Bug reporting form <noreply@postgresql.org>
@ 2026-08-21 14:37 ` Andrey Rachitskiy <pl0h0yp1@gmail.com>
2026-08-21 22:07 ` Re: BUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row Andrey Rachitskiy <pl0h0yp1@gmail.com>
0 siblings, 1 reply; 3+ messages in thread
From: Andrey Rachitskiy @ 2026-08-21 14:37 UTC (permalink / raw)
To: 17801022106@163.com; pgsql-bugs@lists.postgresql.org; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andrew Dunstan <andrew@dunslane.net>
пт, 21 авг. 2026 г. в 17:44, PG Bug reporting form <noreply@postgresql.org>:
> The following bug has been logged on the website:
>
> Bug reference: 19636
> Logged by: anliuan
> Email address: 17801022106@163.com
> PostgreSQL version: 19beta1
> Operating system: centos
> Description:
>
> When a PL/pgSQL function/procedure inserts an array value that arrives as
> an
> expanded object (via EOH_flatten_into), the resulting tuple stores the
> varlena with a 4-byte header instead of converting it to a
> 1-byte short varlena header, wasting 3 bytes per row for arrays <= 127
> bytes total size.
>
> Plain INSERT correctly uses the short varlena format; only the PL/pgSQL
> expanded-object path is affected.
>
> Steps to reproduce:
>
> CREATE TABLE tt2(a text[]);
> INSERT INTO tt2 VALUES('{aaaaa}'); -- plain INSERT
>
> CREATE OR REPLACE FUNCTION call_insert_acl(racl text[]) RETURNS int AS $$
> BEGIN
> INSERT INTO tt2 VALUES(racl);
> RETURN 1;
> END;
> $$ LANGUAGE plpgsql;
>
> SELECT call_insert_acl('{aaaaa}'); -- PL/pgSQL INSERT
>
> SELECT a, pg_column_size(a) FROM tt2;
>
> Expected result: Both rows should have the same pg_column_size (33 bytes —
> short varlena with 1-byte header).
>
> Actual result:
>
> ┌─────────────────┬────────────────┬───────────────────────────────┐
> │ row │ pg_column_size │ header format │
> ├─────────────────┼────────────────┼───────────────────────────────┤
> │ plain INSERT │ 33 │ short varlena (1-byte header) │
> ├─────────────────┼────────────────┼───────────────────────────────┤
> │ PL/pgSQL INSERT │ 36 │ 4-byte header │
> └─────────────────┴────────────────┴───────────────────────────────┘
> Root cause:
>
> In src/backend/access/common/heaptuple.c, heap_fill_tuple has four
> varlena
> paths. The EOH_flatten_into path (for expanded objects) produces a
> 4-byte-header varlena and writes it directly, without checking
> VARATT_CAN_MAKE_SHORT:
>
> /* Current code — line ~229 */
> if (VARATT_IS_EXTERNAL_EXPANDED(val))
> {
> ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
> data = (char *) ATT_ALIGN_NOMINAL(data, att[i]->attalign);
> data_length = EOH_get_flat_size(eoh);
> EOH_flatten_into(eoh, data, data_length);
> /* BUG: no VARATT_CAN_MAKE_SHORT check here */
> }
>
> The normal 4-byte-header path correctly performs this check:
>
> /* Normal path — line ~240 */
> else if (VARLENA_ATT_IS_PACKABLE(att[i]) && VARATT_CAN_MAKE_SHORT(val))
> {
> data_length = VARATT_CONVERTED_SHORT_SIZE(val);
> SET_VARSIZE_SHORT(data, data_length);
> }
>
> heap_compute_data_size has the same omission — it uses
> EOH_get_flat_size()
> without considering the short-header size reduction, so the size
> calculation
> and fill logic are consistently wrong (no memory
> overrun, just wasted space).
>
>
History
-------
Short headers date from 3e23b68dac0 (2007, "Support varlena fields with
single-byte headers and unaligned storage"). heap_form_tuple has packed
inline 4-byte-header varlenas via VARATT_CAN_MAKE_SHORT ever since the
modern heap_form_tuple path.
Expanded objects arrived in 1dc5ebc9077 (2015, "Support expanded
objects, particularly arrays, for better performance"). That commit
added an EXTERNAL_EXPANDED branch in heap_compute_data_size / fill_val
that flattens with EOH_flatten_into and writes the result as-is.
Flatteners are required to produce an inline 4-byte-header varlena
(see expandeddatum.h). The new branch never applied the existing
short-header conversion afterward. So this looks like an omission from
day one of expanded objects, not a later regression.
Proposal Fix
---
In those EXTERNAL_EXPANDED arms, if the attribute is packable and the
flat size from EOH_get_flat_size() would fit a short header, flatten
into a temporary palloc buffer and emit the short form into the tuple.
Otherwise keep the previous path (align and flatten straight into the
tuple).
VARATT_CAN_MAKE_SHORT cannot be used on the expanded toast pointer
itself. It requires a 4B_U varlena. Before flattening we only have
the flat size, so the patch uses a size-only helper matching that
macro's length rule.
The temporary buffer is needed because EOH_flatten_into expects a
maxaligned destination (same constraint as datumSerialize), while short
packing writes at an unaligned data pointer. The temp is not pfree'd.
fill_val can run with CurrentMemoryContext as a BumpContext. On
master / REL_19, RecursiveUnion UNION DISTINCT stores hashed tuples in
a BumpContext tuplescxt (c106ef08071, "Use BumpContext contexts in
TupleHashTables"). LookupTupleHashEntry switches to that context
before ExecCopySlotMinimalTupleExtra → heap_form_minimal_tuple. Bump
does not support pfree. A variant that pfree'd the temp failed the
"with" regress test on 19/master with:
ERROR: pfree is not supported by the bump memory allocator
REL_18 still used AllocSet for that context, so the same pfree passed
there. The short-packable flat size is at most about 130 bytes. The
chunk is reclaimed when the context is reset.
Alternatives considered: Stack buffer was considered (fits the size bound),
but palloc with EOH_flatten_into seemed better to me.
--
Regards,
Rachitskiy Andrey
Attachments:
[text/x-patch] 0001-Pack-short-varlenas-when-flattening-expanded-objects.patch (6.0K, ../../CAB8bMisZBQiCqcVom0wH1m4_JZgkaAkR4xHocSni2fgVjuHnfg@mail.gmail.com/3-0001-Pack-short-varlenas-when-flattening-expanded-objects.patch)
download | inline diff:
From: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Date: Fri, 21 Aug 2026 19:05:00 +0500
Subject: [PATCH] Pack short varlenas when flattening expanded objects into tuples
heap_form_tuple already converts ordinary 4-byte-header varlenas to the
1-byte short header when the value is small enough and the attribute is
packable. The expanded-object path flattened via EOH_flatten_into and
wrote the flattener's mandatory 4-byte-header result directly, so small
arrays (and other expanded types) inserted from PL/pgSQL kept a 4-byte
header and wasted three bytes per value compared with a plain INSERT of
the same datum.
After flattening, apply the same short-header conversion when the flat
size fits. Flatten into a temporary palloc buffer first because
flatten_into expects a maxaligned destination while short packing does
not. Do not pfree it, since CurrentMemoryContext may be a bump
allocator.
Author: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Reported-by: anliuan <17801022106@163.com>
Discussion: https://www.postgresql.org/message-id/19636-c44abe40ca7e2c4d@postgresql.org
---
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index f30346469ed..693d7c751eb 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -87,6 +87,16 @@
#define COMPACT_ATTR_IS_PACKABLE(att) \
((att)->attlen == -1 && (att)->attispackable)
+/*
+ * Size rule matching VARATT_CAN_MAKE_SHORT, for use with EOH_get_flat_size()
+ * before flattening. VARATT_CAN_MAKE_SHORT needs a 4-byte-header (4B_U)
+ * varlena. An expanded-object toast pointer is not one.
+ */
+#define VARATT_SIZE_CAN_MAKE_SHORT(len) \
+ ((len) - VARHDRSZ + VARHDRSZ_SHORT <= VARATT_SHORT_MAX)
+#define VARATT_SHORT_SIZE_FROM_4B(len) \
+ ((len) - VARHDRSZ + VARHDRSZ_SHORT)
+
/*
* Setup for caching pass-by-ref missing attributes in a way that survives
* tupleDesc destruction.
@@ -248,11 +258,20 @@ heap_compute_data_size(TupleDesc tupleDesc,
VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(val)))
{
/*
- * we want to flatten the expanded value so that the constructed
- * tuple doesn't depend on it
+ * Expanded objects will be flattened into a 4-byte-header
+ * varlena. If that fits a short header, account for packing
+ * (no alignment), matching fill_val.
*/
- data_length = att_nominal_alignby(data_length, atti->attalignby);
- data_length += EOH_get_flat_size(DatumGetEOHP(val));
+ Size flat_size = EOH_get_flat_size(DatumGetEOHP(val));
+
+ if (COMPACT_ATTR_IS_PACKABLE(atti) &&
+ VARATT_SIZE_CAN_MAKE_SHORT(flat_size))
+ data_length += VARATT_SHORT_SIZE_FROM_4B(flat_size);
+ else
+ {
+ data_length = att_nominal_alignby(data_length, atti->attalignby);
+ data_length += flat_size;
+ }
}
else
{
@@ -329,14 +348,36 @@ fill_val(CompactAttribute *att,
if (VARATT_IS_EXTERNAL_EXPANDED(val))
{
/*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
+ * Flatten so the tuple doesn't depend on the expanded
+ * object. Flatteners produce a 4-byte-header varlena.
+ * Convert to short header when possible.
*/
ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
-
- data = (char *) att_nominal_alignby(data, att->attalignby);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
+ Size flat_size = EOH_get_flat_size(eoh);
+
+ if (att->attispackable &&
+ VARATT_SIZE_CAN_MAKE_SHORT(flat_size))
+ {
+ char *tmp;
+
+ /*
+ * Flatten into a temp buffer: EOH_flatten_into needs a
+ * maxaligned destination, short packing does not.
+ * Do not pfree(tmp); CurrentMemoryContext may be a bump
+ * allocator.
+ */
+ tmp = palloc(flat_size);
+ EOH_flatten_into(eoh, tmp, flat_size);
+ data_length = VARATT_CONVERTED_SHORT_SIZE(tmp);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(tmp), data_length - 1);
+ }
+ else
+ {
+ data = (char *) att_nominal_alignby(data, att->attalignby);
+ data_length = flat_size;
+ EOH_flatten_into(eoh, data, data_length);
+ }
}
else
{
diff --git a/src/test/regress/expected/plpgsql.out b/src/test/regress/expected/plpgsql.out
index b37b2abaf80..3def42938d8 100644
--- a/src/test/regress/expected/plpgsql.out
+++ b/src/test/regress/expected/plpgsql.out
@@ -5161,6 +5161,30 @@ begin
raise notice 'a = %', a;
end$$;
NOTICE: a = {1,2,3}
+-- Inserting an expanded array should use the same short-varlena packing as a
+-- plain INSERT of the equivalent flat value.
+create temp table expanded_short_pack(a text[]);
+insert into expanded_short_pack values ('{aaaaa}');
+create function insert_expanded_short_pack(racl text[]) returns void as $$
+begin
+ insert into expanded_short_pack values (racl);
+end;
+$$ language plpgsql;
+select insert_expanded_short_pack('{aaaaa}');
+ insert_expanded_short_pack
+----------------------------
+
+(1 row)
+
+select pg_column_size(a) from expanded_short_pack;
+ pg_column_size
+----------------
+ 33
+ 33
+(2 rows)
+
+drop function insert_expanded_short_pack(text[]);
+drop table expanded_short_pack;
--
-- Test access to call stack
--
diff --git a/src/test/regress/sql/plpgsql.sql b/src/test/regress/sql/plpgsql.sql
index ae6b67e3e22..e97d9a686c5 100644
--- a/src/test/regress/sql/plpgsql.sql
+++ b/src/test/regress/sql/plpgsql.sql
@@ -4160,6 +4160,20 @@ begin
raise notice 'a = %', a;
end$$;
+-- Inserting an expanded array should use the same short-varlena packing as a
+-- plain INSERT of the equivalent flat value.
+create temp table expanded_short_pack(a text[]);
+insert into expanded_short_pack values ('{aaaaa}');
+create function insert_expanded_short_pack(racl text[]) returns void as $$
+begin
+ insert into expanded_short_pack values (racl);
+end;
+$$ language plpgsql;
+select insert_expanded_short_pack('{aaaaa}');
+select pg_column_size(a) from expanded_short_pack;
+drop function insert_expanded_short_pack(text[]);
+drop table expanded_short_pack;
+
--
-- Test access to call stack
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: BUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row
2026-08-21 09:23 BUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row PG Bug reporting form <noreply@postgresql.org>
2026-08-21 14:37 ` Re: BUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row Andrey Rachitskiy <pl0h0yp1@gmail.com>
@ 2026-08-21 22:07 ` Andrey Rachitskiy <pl0h0yp1@gmail.com>
0 siblings, 0 replies; 3+ messages in thread
From: Andrey Rachitskiy @ 2026-08-21 22:07 UTC (permalink / raw)
To: 17801022106@163.com; pgsql-bugs@lists.postgresql.org; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andrew Dunstan <andrew@dunslane.net>
пт, 21 авг. 2026 г. в 19:37, Andrey Rachitskiy <pl0h0yp1@gmail.com>:
> History
> -------
> Short headers date from 3e23b68dac0 (2007, "Support varlena fields with
> single-byte headers and unaligned storage"). heap_form_tuple has packed
> inline 4-byte-header varlenas via VARATT_CAN_MAKE_SHORT ever since the
> modern heap_form_tuple path.
>
> Expanded objects arrived in 1dc5ebc9077 (2015, "Support expanded
> objects, particularly arrays, for better performance"). That commit
> added an EXTERNAL_EXPANDED branch in heap_compute_data_size / fill_val
> that flattens with EOH_flatten_into and writes the result as-is.
> Flatteners are required to produce an inline 4-byte-header varlena
> (see expandeddatum.h). The new branch never applied the existing
> short-header conversion afterward. So this looks like an omission from
> day one of expanded objects, not a later regression.
>
> Proposal Fix
> ---
> In those EXTERNAL_EXPANDED arms, if the attribute is packable and the
> flat size from EOH_get_flat_size() would fit a short header, flatten
> into a temporary palloc buffer and emit the short form into the tuple.
> Otherwise keep the previous path (align and flatten straight into the
> tuple).
>
> VARATT_CAN_MAKE_SHORT cannot be used on the expanded toast pointer
> itself. It requires a 4B_U varlena. Before flattening we only have
> the flat size, so the patch uses a size-only helper matching that
> macro's length rule.
>
> The temporary buffer is needed because EOH_flatten_into expects a
> maxaligned destination (same constraint as datumSerialize), while short
> packing writes at an unaligned data pointer. The temp is not pfree'd.
> fill_val can run with CurrentMemoryContext as a BumpContext. On
> master / REL_19, RecursiveUnion UNION DISTINCT stores hashed tuples in
> a BumpContext tuplescxt (c106ef08071, "Use BumpContext contexts in
> TupleHashTables"). LookupTupleHashEntry switches to that context
> before ExecCopySlotMinimalTupleExtra → heap_form_minimal_tuple. Bump
> does not support pfree. A variant that pfree'd the temp failed the
> "with" regress test on 19/master with:
>
> ERROR: pfree is not supported by the bump memory allocator
>
> REL_18 still used AllocSet for that context, so the same pfree passed
> there. The short-packable flat size is at most about 130 bytes. The
> chunk is reclaimed when the context is reset.
>
> Alternatives considered: Stack buffer was considered (fits the size
> bound), but palloc with EOH_flatten_into seemed better to me.
>
>
A follow-up on the repro and on the regress in v1.
The heaptuple omission is real, but the reporter's one-shot example often
does not reach it. With the default plan_cache_mode = auto, the first
executions of the PL/pgSQL INSERT use a custom plan. The bound array
parameter is substituted during planning. datumCopy flattens the
expanded object there, so fill_val later sees an ordinary 4-byte-header
varlena and packs it to short as usual. Both rows then show
pg_column_size 33 even without the fix.
I checked both cases with gdb on an unpatched build, breaking on
EOH_flatten_into.
1. Default plan_cache_mode (auto), one-shot PL/pgSQL INSERT — custom plan
EOH_flatten_into is called once, with allocated_size 36, from datumCopy
during planning (parameter substitution in eval_const_expressions):
#0 EOH_flatten_into (... allocated_size=36)
#1 datumCopy
#3 eval_const_expressions_mutator
...
# BuildCachedPlan / GetCachedPlan
# SPI_execute_plan_with_paramlist
# plpgsql exec_stmt_execsql
There is no fill_val frame on that hit. heap_form_tuple later sees an
ordinary flat 4-byte-header varlena and applies VARATT_CAN_MAKE_SHORT as
usual. pg_column_size is 33 for the plain INSERT and 33 for the
PL/pgSQL INSERT.
2. Same SQL with a generic plan
SET plan_cache_mode = force_generic_plan;
EOH_flatten_into is again called with allocated_size 36, but the caller
is the EXTERNAL_EXPANDED arm in fill_val:
#0 EOH_flatten_into (... allocated_size=36)
#1 fill_val (... heaptuple.c)
#2 heap_fill_tuple
#3 heap_form_tuple
...
# ExecModifyTable / SPI / plpgsql
That arm writes the flattener output as-is, so the PL/pgSQL row keeps
the 4-byte header. Without the fix pg_column_size is 33 (plain INSERT)
and 36 (PL/pgSQL). With the fix both are 33.
Under plan_cache_mode = auto the same fill_val path appears after the
plan cache switches from custom to generic (after several executions of
the INSERT). force_generic_plan just makes that path reliable on the
first call.
v2 of the patch is attached. The heaptuple change is the same as v1.
The plpgsql regress now sets force_generic_plan around the test so case
2 is what we cover. Case 1 would pass even without the fix.
--
Regards,
Rachitskiy Andrey
Attachments:
[text/x-patch] v2-0001-Pack-short-varlenas-when-flattening-expanded-objects.patch (6.7K, ../../CAB8bMivoqD0ScXgr3yv-Sq=8jVsGSS+UaEYA+1hs1WBTsm1kvQ@mail.gmail.com/3-v2-0001-Pack-short-varlenas-when-flattening-expanded-objects.patch)
download | inline diff:
From bd22c82edaa05e1d87d73e8ac0e43391a94163a3 Mon Sep 17 00:00:00 2001
From: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Date: Sat, 22 Aug 2026 02:19:08 +0500
Subject: [PATCH] Pack short varlenas when flattening expanded objects into
tuples
heap_form_tuple already converts ordinary 4-byte-header varlenas to the
1-byte short header when the value is small enough and the attribute is
packable. The expanded-object path flattened via EOH_flatten_into and
wrote the flattener's mandatory 4-byte-header result directly, so small
arrays (and other expanded types) inserted from PL/pgSQL kept a 4-byte
header and wasted three bytes per value compared with a plain INSERT of
the same datum.
After flattening, apply the same short-header conversion when the flat
size fits. Flatten into a temporary palloc buffer first because
flatten_into expects a maxaligned destination while short packing does
not. Do not pfree it, since CurrentMemoryContext may be a bump
allocator.
The plpgsql regress forces a generic plan so the parameter remains an
expanded object through to heap_form_tuple. A custom plan flattens via
datumCopy during planning and would not exercise the EXTERNAL_EXPANDED
arms.
Author: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Reported-by: anliuan <17801022106@163.com>
Discussion: https://www.postgresql.org/message-id/19636-c44abe40ca7e2c4d@postgresql.org
---
src/backend/access/common/heaptuple.c | 61 ++++++++++++++++++++++-----
src/test/regress/expected/plpgsql.out | 28 ++++++++++++
src/test/regress/sql/plpgsql.sql | 18 ++++++++
3 files changed, 97 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index f30346469ed..d1f5a09a286 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -87,6 +87,16 @@
#define COMPACT_ATTR_IS_PACKABLE(att) \
((att)->attlen == -1 && (att)->attispackable)
+/*
+ * Size rule matching VARATT_CAN_MAKE_SHORT, for use with EOH_get_flat_size()
+ * before flattening. VARATT_CAN_MAKE_SHORT needs a 4-byte-header (4B_U)
+ * varlena. An expanded-object toast pointer is not one.
+ */
+#define VARATT_SIZE_CAN_MAKE_SHORT(len) \
+ ((len) - VARHDRSZ + VARHDRSZ_SHORT <= VARATT_SHORT_MAX)
+#define VARATT_SHORT_SIZE_FROM_4B(len) \
+ ((len) - VARHDRSZ + VARHDRSZ_SHORT)
+
/*
* Setup for caching pass-by-ref missing attributes in a way that survives
* tupleDesc destruction.
@@ -248,11 +258,20 @@ heap_compute_data_size(TupleDesc tupleDesc,
VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(val)))
{
/*
- * we want to flatten the expanded value so that the constructed
- * tuple doesn't depend on it
+ * Expanded objects will be flattened into a 4-byte-header
+ * varlena. If that fits a short header, account for packing
+ * (no alignment), matching fill_val.
*/
- data_length = att_nominal_alignby(data_length, atti->attalignby);
- data_length += EOH_get_flat_size(DatumGetEOHP(val));
+ Size flat_size = EOH_get_flat_size(DatumGetEOHP(val));
+
+ if (COMPACT_ATTR_IS_PACKABLE(atti) &&
+ VARATT_SIZE_CAN_MAKE_SHORT(flat_size))
+ data_length += VARATT_SHORT_SIZE_FROM_4B(flat_size);
+ else
+ {
+ data_length = att_nominal_alignby(data_length, atti->attalignby);
+ data_length += flat_size;
+ }
}
else
{
@@ -329,14 +348,36 @@ fill_val(CompactAttribute *att,
if (VARATT_IS_EXTERNAL_EXPANDED(val))
{
/*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
+ * Flatten so the tuple doesn't depend on the expanded
+ * object. Flatteners produce a 4-byte-header varlena.
+ * Convert to short header when possible.
*/
ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
-
- data = (char *) att_nominal_alignby(data, att->attalignby);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
+ Size flat_size = EOH_get_flat_size(eoh);
+
+ if (COMPACT_ATTR_IS_PACKABLE(att) &&
+ VARATT_SIZE_CAN_MAKE_SHORT(flat_size))
+ {
+ char *tmp;
+
+ /*
+ * Flatten into a temp buffer: EOH_flatten_into needs a
+ * maxaligned destination, short packing does not.
+ * Do not pfree(tmp); CurrentMemoryContext may be a bump
+ * allocator.
+ */
+ tmp = palloc(flat_size);
+ EOH_flatten_into(eoh, tmp, flat_size);
+ data_length = VARATT_CONVERTED_SHORT_SIZE(tmp);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(tmp), data_length - 1);
+ }
+ else
+ {
+ data = (char *) att_nominal_alignby(data, att->attalignby);
+ data_length = flat_size;
+ EOH_flatten_into(eoh, data, data_length);
+ }
}
else
{
diff --git a/src/test/regress/expected/plpgsql.out b/src/test/regress/expected/plpgsql.out
index b37b2abaf80..69fbbc87ac4 100644
--- a/src/test/regress/expected/plpgsql.out
+++ b/src/test/regress/expected/plpgsql.out
@@ -5161,6 +5161,32 @@ begin
raise notice 'a = %', a;
end$$;
NOTICE: a = {1,2,3}
+-- Inserting an expanded array should use the same short-varlena packing as a
+-- plain INSERT of the equivalent flat value.
+set plan_cache_mode = force_generic_plan;
+create temp table expanded_short_pack(a text[]);
+insert into expanded_short_pack values ('{aaaaa}');
+create function insert_expanded_short_pack(racl text[]) returns void as $$
+begin
+ insert into expanded_short_pack values (racl);
+end;
+$$ language plpgsql;
+select insert_expanded_short_pack('{aaaaa}');
+ insert_expanded_short_pack
+----------------------------
+
+(1 row)
+
+select pg_column_size(a) from expanded_short_pack;
+ pg_column_size
+----------------
+ 33
+ 33
+(2 rows)
+
+drop function insert_expanded_short_pack(text[]);
+drop table expanded_short_pack;
+reset plan_cache_mode;
--
-- Test access to call stack
--
diff --git a/src/test/regress/sql/plpgsql.sql b/src/test/regress/sql/plpgsql.sql
index ae6b67e3e22..cadb8398375 100644
--- a/src/test/regress/sql/plpgsql.sql
+++ b/src/test/regress/sql/plpgsql.sql
@@ -4160,6 +4160,22 @@ begin
raise notice 'a = %', a;
end$$;
+-- Inserting an expanded array should use the same short-varlena packing as a
+-- plain INSERT of the equivalent flat value.
+set plan_cache_mode = force_generic_plan;
+create temp table expanded_short_pack(a text[]);
+insert into expanded_short_pack values ('{aaaaa}');
+create function insert_expanded_short_pack(racl text[]) returns void as $$
+begin
+ insert into expanded_short_pack values (racl);
+end;
+$$ language plpgsql;
+select insert_expanded_short_pack('{aaaaa}');
+select pg_column_size(a) from expanded_short_pack;
+drop function insert_expanded_short_pack(text[]);
+drop table expanded_short_pack;
+reset plan_cache_mode;
+
--
-- Test access to call stack
--
2.53.0
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2026-08-21 22:07 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-08-21 09:23 BUG #19636: heap_fill_tuple misses short varlena conversion after EOH_flatten_into, causing 3-byte waste per row PG Bug reporting form <noreply@postgresql.org>
2026-08-21 14:37 ` Andrey Rachitskiy <pl0h0yp1@gmail.com>
2026-08-21 22:07 ` Andrey Rachitskiy <pl0h0yp1@gmail.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox