From: Chao Li <li.evan.chao@gmail.com>
Message-Id: <BB69B882-5401-45CE-919E-8A78511EDABC@gmail.com>
Content-Type: multipart/mixed;
	boundary="Apple-Mail=_4BC10C56-98F9-470C-861E-830388C69665"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.600.51.1.1\))
Subject: Re: Fix tuple deformation with virtual generated NOT NULL columns
Date: Tue, 9 Jun 2026 10:19:25 +0800
In-Reply-To: <pse3eru75b5skbvc7jjrf7origavqojc6nqtwdrr7z6sjkyxfo@siklvj643v4f>
Cc: David Rowley <dgrowleyml@gmail.com>,
 Postgres hackers <pgsql-hackers@lists.postgresql.org>
To: Andres Freund <andres@anarazel.de>
References: <A4BC563C-0CA3-4EF3-952A-EA41F9E5BF1E@gmail.com>
 <CAApHDvoQKY8zHb1LZBuYZBRszi0qVmTaV_zFup=A9xqRpJWMRQ@mail.gmail.com>
 <pse3eru75b5skbvc7jjrf7origavqojc6nqtwdrr7z6sjkyxfo@siklvj643v4f>
Archived-At: 
 <https://www.postgresql.org/message-id/BB69B882-5401-45CE-919E-8A78511EDABC%40gmail.com>
Precedence: bulk


--Apple-Mail=_4BC10C56-98F9-470C-861E-830388C69665
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8


> On Jun 8, 2026, at 21:01, Andres Freund <andres@anarazel.de> wrote:
>=20
> Hi,
>=20
> On 2026-06-06 16:50:29 +1200, David Rowley wrote:
>> On Thu, 4 Jun 2026 at 17:57, Chao Li <li.evan.chao@gmail.com> wrote:
>>> While testing "Optimize tuple deformation=E2=80=9D, I found a bug:
>>=20
>>> I think the problem is in finding the first non-guaranteed attribute =
where virtual generated attributes are not considered:
>>=20
>> Thanks for the report and fix. I pushed a slightly adjusted version.
>> Form_pg_attribute.attgenerated is '\0' for non-generated columns, so
>> there's no point in checking cattr->attgenerated as well as that.
>>=20
>> I also added an Assert() to help catch any other reason that the
>> guaranteed column gets set incorrectly again in TupleDescFinalize().
>=20
> Seems like a test for some of this would be good too?
>=20
> Greetings,
>=20
> Andres Freund

Okay, I tried to add a test matching my repro. With this test in place, =
I reverted the fix and ran make check; it failed by hitting the Assert =
David added:
```
TRAP: failed Assert("first_null_attr(tup->t_bits, natts) >=3D =
firstNullAttr"), File: "execTuples.c", Line: 1083, PID: 65804
0   postgres                            0x0000000104e883b8 =
ExceptionalCondition + 216
1   postgres                            0x00000001048e5ebc =
slot_deform_heap_tuple + 456
2   postgres                            0x00000001048e2a18 =
tts_buffer_heap_getsomeattrs + 112
3   postgres                            0x00000001048c5044 =
slot_getsomeattrs + 68
4   postgres                            0x00000001048b97a4 =
ExecInterpExpr + 416
5   postgres                            0x00000001048b8ea0 =
ExecInterpExprStillValid + 76
6   postgres                            0x000000010492ec68 =
ExecEvalExprNoReturn + 44
7   postgres                            0x000000010492ec28 =
ExecEvalExprNoReturnSwitchContext + 48
8   postgres                            0x000000010492eb20 ExecProject + =
72
9   postgres                            0x000000010492e680 =
ExecScanExtended + 288
10  postgres                            0x000000010492d7cc =
ExecSeqScanWithProject + 220
11  postgres                            0x00000001048db508 =
ExecProcNodeFirst + 92
12  postgres                            0x00000001048d1eb8 ExecProcNode =
+ 60
```

See the attached patch for details.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/


--Apple-Mail=_4BC10C56-98F9-470C-861E-830388C69665
Content-Disposition: attachment;
	filename=v1-0001-Add-regression-test-for-virtual-generated-column-.patch
Content-Type: application/octet-stream;
	x-unix-mode=0644;
	name="v1-0001-Add-regression-test-for-virtual-generated-column-.patch"
Content-Transfer-Encoding: quoted-printable

=46rom=20944e72ad211a9ac694e7fdeb9361f9fe3e2a01d4=20Mon=20Sep=2017=20=
00:00:00=202001=0AFrom:=20"Chao=20Li=20(Evan)"=20<lic@highgo.com>=0A=
Date:=20Tue,=209=20Jun=202026=2009:58:44=20+0800=0ASubject:=20[PATCH=20=
v1]=20Add=20regression=20test=20for=20virtual=20generated=20column=0A=20=
deformation=0A=0AAdd=20coverage=20for=20a=20virtual=20generated=20NOT=20=
NULL=20column=20followed=20by=20a=0Aphysically=20stored=20NOT=20NULL=20=
column.=20This=20exercises=20the=20tuple=20deformation=0Acase=20fixed=20=
by=2089eafad297a,=20where=20TupleDescFinalize()=20could=20incorrectly=0A=
treat=20a=20virtual=20generated=20column=20as=20part=20of=20the=20=
guaranteed=20physical=20column=0Aprefix=20and=20compute=20cached=20=
offsets=20past=20it.=0A=0AWithout=20that=20fix,=20deforming=20the=20=
following=20column=20could=20read=20from=20the=20wrong=0Atuple=20offset.=0A=
=0ASuggested-by:=20Andres=20Freund=20<andres@anarazel.de>=0AAuthor:=20=
Chao=20Li=20<lic@highgo.com>=0ADiscussion:=20=
https://postgr.es/m/A4BC563C-0CA3-4EF3-952A-EA41F9E5BF1E%40gmail.com=0A=
---=0A=20src/test/regress/expected/generated_virtual.out=20|=209=20=
+++++++++=0A=20src/test/regress/sql/generated_virtual.sql=20=20=20=20=20=20=
|=205=20+++++=0A=202=20files=20changed,=2014=20insertions(+)=0A=0Adiff=20=
--git=20a/src/test/regress/expected/generated_virtual.out=20=
b/src/test/regress/expected/generated_virtual.out=0Aindex=20=
24d5dbf46ca..7a5788146f5=20100644=0A---=20=
a/src/test/regress/expected/generated_virtual.out=0A+++=20=
b/src/test/regress/expected/generated_virtual.out=0A@@=20-727,6=20=
+727,15=20@@=20ERROR:=20=20null=20value=20in=20column=20"b"=20of=20=
relation=20"gtest21b"=20violates=20not-null=20constr=0A=20DETAIL:=20=20=
Failing=20row=20contains=20(null,=20virtual).=0A=20ALTER=20TABLE=20=
gtest21b=20ALTER=20COLUMN=20b=20DROP=20NOT=20NULL;=0A=20INSERT=20INTO=20=
gtest21b=20(a)=20VALUES=20(0);=20=20--=20ok=20now=0A+--=20virtual=20=
generated=20columns=20are=20not=20physically=20stored,=20even=20when=20=
not=20null=0A+CREATE=20TABLE=20gtest21c=20(a=20int=20NOT=20NULL,=20b=20=
int=20GENERATED=20ALWAYS=20AS=20(a=20*=202)=20VIRTUAL=20NOT=20NULL,=20c=20=
int=20NOT=20NULL);=0A+INSERT=20INTO=20gtest21c=20(a,=20c)=20VALUES=20=
(10,=2042);=0A+SELECT=20a,=20b,=20c=20FROM=20gtest21c;=0A+=20a=20=20|=20=
b=20=20|=20c=20=20=0A+----+----+----=0A+=2010=20|=2020=20|=2042=0A+(1=20=
row)=0A+=0A=20--=20not-null=20constraint=20with=20partitioned=20table=0A=20=
CREATE=20TABLE=20gtestnn_parent=20(=0A=20=20=20=20=20f1=20int,=0Adiff=20=
--git=20a/src/test/regress/sql/generated_virtual.sql=20=
b/src/test/regress/sql/generated_virtual.sql=0Aindex=20=
9c2bb6590b3..126ae3ecda9=20100644=0A---=20=
a/src/test/regress/sql/generated_virtual.sql=0A+++=20=
b/src/test/regress/sql/generated_virtual.sql=0A@@=20-374,6=20+374,11=20=
@@=20INSERT=20INTO=20gtest21b=20(a)=20VALUES=20(NULL);=20=20--=20error=0A=
=20ALTER=20TABLE=20gtest21b=20ALTER=20COLUMN=20b=20DROP=20NOT=20NULL;=0A=20=
INSERT=20INTO=20gtest21b=20(a)=20VALUES=20(0);=20=20--=20ok=20now=0A=20=0A=
+--=20virtual=20generated=20columns=20are=20not=20physically=20stored,=20=
even=20when=20not=20null=0A+CREATE=20TABLE=20gtest21c=20(a=20int=20NOT=20=
NULL,=20b=20int=20GENERATED=20ALWAYS=20AS=20(a=20*=202)=20VIRTUAL=20NOT=20=
NULL,=20c=20int=20NOT=20NULL);=0A+INSERT=20INTO=20gtest21c=20(a,=20c)=20=
VALUES=20(10,=2042);=0A+SELECT=20a,=20b,=20c=20FROM=20gtest21c;=0A+=0A=20=
--=20not-null=20constraint=20with=20partitioned=20table=0A=20CREATE=20=
TABLE=20gtestnn_parent=20(=0A=20=20=20=20=20f1=20int,=0A--=20=0A2.50.1=20=
(Apple=20Git-155)=0A=0A=

--Apple-Mail=_4BC10C56-98F9-470C-861E-830388C69665--