MIME-Version: 1.0
From: SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>
Date: Mon, 4 May 2026 02:17:08 -0700
Message-ID: 
 <CAHg+QDeoccU5CudrJpmSKZfKZ1gRMNY=5BxSC=JpHgkonzgcOw@mail.gmail.com>
Subject: [Bug]Vacuum full silently NULL out fast default columns
To: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
	=?UTF-8?Q?=C3=81lvaro_Herrera?= <alvherre@kurilemu.de>,
	Antonin Houska <ah@cybertec.at>
Content-Type: multipart/mixed; boundary="000000000000c64b5b0650fa65bd"
Archived-At: 
 <https://www.postgresql.org/message-id/CAHg%2BQDeoccU5CudrJpmSKZfKZ1gRMNY%3D5BxSC%3DJpHgkonzgcOw%40mail.gmail.com>
Precedence: bulk

--000000000000c64b5b0650fa65bd
Content-Type: multipart/alternative; boundary="000000000000c64b590650fa65bb"

--000000000000c64b590650fa65bb
Content-Type: text/plain; charset="UTF-8"

Hi Hackers,

VACUUM FULL silently turns columns added via ALTER TABLE ... ADD COLUMN ...
DEFAULT <const> into NULL
on all pre-existing rows. The issue exists for other operations like
CLUSTER, REPACK.

Repro:

CREATE TABLE t (id int PRIMARY KEY);
INSERT INTO t SELECT generate_series(1,3);
ALTER TABLE t ADD COLUMN x int DEFAULT 42;
SELECT * FROM t;     -- (1,42),(2,42),(3,42)
VACUUM FULL t;
SELECT * FROM t;     -- (1,NULL),(2,NULL),(3,NULL)

If the column is NOT NULL, the value becomes the type's zero value
instead of NULL, silently bypassing both NOT NULL and any CHECK
constraint declared on it.

Root Cause: fast path in reform_tuple() in heapam_handler.c returns a copy
of the source tuple when no dropped columns need fixing up. The check
doesn't account for short tuples (HeapTupleHeaderGetNatts(t) <
relnatts) that rely on attmissingval to materialize the default. After
the rewrite, finish_heap_swap() calls RelationClearMissing(), clearing
the only source of those values, and the short tuples then read as
NULL.


Fix: force reform when the source tuple is shorter than the new tuple
descriptor.

Patch attached. Added a regression test in fast_default.sql covering
VACUUM FULL, CLUSTER, and REPACK on a table with fast-default columns
including a NOT NULL CHECK column.

Thanks,
Satya

--000000000000c64b590650fa65bb
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Hackers,<div><br></div><div>VACUUM FULL silently turns =
columns added via ALTER TABLE ... ADD COLUMN ... DEFAULT &lt;const&gt; into=
 NULL <br>on all pre-existing rows. The issue exists for other operations l=
ike CLUSTER, REPACK. <br><br>Repro:<br><br>CREATE TABLE t (id int PRIMARY K=
EY);<br>INSERT INTO t SELECT generate_series(1,3);<br>ALTER TABLE t ADD COL=
UMN x int DEFAULT 42;<br>SELECT * FROM t; =C2=A0 =C2=A0 -- (1,42),(2,42),(3=
,42)<br>VACUUM FULL t;<br>SELECT * FROM t; =C2=A0 =C2=A0 -- (1,NULL),(2,NUL=
L),(3,NULL)<br><br>If the column is NOT NULL, the value becomes the type=
9;s zero value<br>instead of NULL, silently bypassing both NOT NULL and any=
 CHECK<br>constraint declared on it.<br><br>Root Cause: fast path in reform=
_tuple() in heapam_handler.c returns a copy<br>of the source tuple when no =
dropped columns need fixing up. The check<br>doesn&#39;t account for short =
tuples (HeapTupleHeaderGetNatts(t) &lt;<br>relnatts) that rely on attmissin=
gval to materialize the default. After<br>the rewrite, finish_heap_swap() c=
alls RelationClearMissing(), clearing<br>the only source of those values, a=
nd the short tuples then read as<br>NULL.<br><br><br>Fix: force reform when=
 the source tuple is shorter than the new tuple<br>descriptor.<br><br>Patch=
 attached. Added a regression test in fast_default.sql covering<br>VACUUM F=
ULL, CLUSTER, and REPACK on a table with fast-default columns<br>including =
a NOT NULL CHECK column.<br><br>Thanks,<br>Satya</div></div>

--000000000000c64b590650fa65bb--

--000000000000c64b5b0650fa65bd
Content-Type: application/octet-stream;
	name="0001-VACUUM-FULL-silently-NULL-out-fast-default-columns.patch"
Content-Disposition: attachment;
	filename="0001-VACUUM-FULL-silently-NULL-out-fast-default-columns.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_moqzh3tr0>
X-Attachment-Id: f_moqzh3tr0

RnJvbSAxYTNlMjU4NTU0NTIxMzdkNmQ3YzZkYjg5YTI0M2ZmODA4OWU5MzlmIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBTYXR5YSBOYXJsYXB1cmFtIDxzYXR5YW5hcmxhcHVyYW1AZ21h
aWwuY29tPgpEYXRlOiBNb24sIDQgTWF5IDIwMjYgMDg6NTI6NTAgKzAwMDAKU3ViamVjdDogW1BB
VENIXSBWQUNVVU0gRlVMTCBzaWxlbnRseSBOVUxMIG91dCBmYXN0LWRlZmF1bHQgY29sdW1ucwoK
Q29tbWl0IDI4ZDUzNGUyIGludHJvZHVjZWQgcmVmb3JtX3R1cGxlKCkgaW4gaGVhcGFtX2hhbmRs
ZXIuYyB3aXRoIGEKZmFzdCBwYXRoIHRoYXQgcmV0dXJucyB0aGUgc291cmNlIHR1cGxlIHZlcmJh
dGltIHdoZW4gbm8gZHJvcHBlZApjb2x1bW5zIHJlcXVpcmUgZml4aW5nIHVwLiAgVGhlIGNoZWNr
IGlnbm9yZXMgdHVwbGVzIHRoYXQgYXJlIHNob3J0CmR1ZSB0byBhdHRtaXNzaW5ndmFsICh0aGUg
ZmFzdC1kZWZhdWx0IG1lY2hhbmlzbSkuCgpBZnRlciB0aGUgcmV3cml0ZSwgZmluaXNoX2hlYXBf
c3dhcCgpIGNhbGxzIFJlbGF0aW9uQ2xlYXJNaXNzaW5nKCksCmNsZWFyaW5nIHRoZSBjYXRhbG9n
IG1ldGFkYXRhIHRoYXQgd2FzIHRoZSBvbmx5IHNvdXJjZSBvZiB0aG9zZQp2YWx1ZXMuICBTaG9y
dCB0dXBsZXMgdGhlbiByZWFkIGFzIE5VTEwgKG9yIHRoZSB0eXBlJ3MgemVybyB2YWx1ZSBpZgpO
T1QgTlVMTCBpcyBpbiBlZmZlY3QsIGFsc28gYnlwYXNzaW5nIENIRUNLIGNvbnN0cmFpbnRzKS4g
IEFmZmVjdHMKVkFDVVVNIEZVTEwsIENMVVNURVIsIFJFUEFDSywgYW5kIFJFUEFDSyAoQ09OQ1VS
UkVOVExZKS4KCkZvcmNlIHJlZm9ybSB3aGVuIHRoZSBzb3VyY2UgdHVwbGUgaXMgc2hvcnRlciB0
aGFuIHRoZSBuZXcgdHVwbGUKZGVzY3JpcHRvciBzbyBoZWFwX2RlZm9ybV90dXBsZSgpIG1hdGVy
aWFsaXplcyB0aGUgbWlzc2luZyB2YWx1ZXMKYmVmb3JlIGhlYXBfZm9ybV90dXBsZSgpIHJlYnVp
bGRzIGEgZnVsbC13aWR0aCB0dXBsZS4KLS0tCiBzcmMvYmFja2VuZC9hY2Nlc3MvaGVhcC9oZWFw
YW1faGFuZGxlci5jICAgfCAxMiArKysrKysrLQogc3JjL3Rlc3QvcmVncmVzcy9leHBlY3RlZC9m
YXN0X2RlZmF1bHQub3V0IHwgMzYgKysrKysrKysrKysrKysrKysrKysrKwogc3JjL3Rlc3QvcmVn
cmVzcy9zcWwvZmFzdF9kZWZhdWx0LnNxbCAgICAgIHwgMTYgKysrKysrKysrKwogMyBmaWxlcyBj
aGFuZ2VkLCA2MyBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCgpkaWZmIC0tZ2l0IGEvc3Jj
L2JhY2tlbmQvYWNjZXNzL2hlYXAvaGVhcGFtX2hhbmRsZXIuYyBiL3NyYy9iYWNrZW5kL2FjY2Vz
cy9oZWFwL2hlYXBhbV9oYW5kbGVyLmMKaW5kZXggMjBkM2I0NmUuLjEyYjI3Nzc2IDEwMDY0NAot
LS0gYS9zcmMvYmFja2VuZC9hY2Nlc3MvaGVhcC9oZWFwYW1faGFuZGxlci5jCisrKyBiL3NyYy9i
YWNrZW5kL2FjY2Vzcy9oZWFwL2hlYXBhbV9oYW5kbGVyLmMKQEAgLTI0MTEsOCArMjQxMSwxOCBA
QCByZWZvcm1fdHVwbGUoSGVhcFR1cGxlIHR1cGxlLCBSZWxhdGlvbiBPbGRIZWFwLCBSZWxhdGlv
biBOZXdIZWFwLAogCVR1cGxlRGVzYwluZXdUdXBEZXNjID0gUmVsYXRpb25HZXREZXNjcihOZXdI
ZWFwKTsKIAlib29sCQluZWVkc19yZWZvcm0gPSBmYWxzZTsKIAorCS8qCisJICogSWYgdGhlIHR1
cGxlIGlzIHNob3J0IGR1ZSB0byBtaXNzaW5nLXZhbHVlIChmYXN0LWRlZmF1bHQpIGF0dHJpYnV0
ZXMsCisJICogd2UgbXVzdCBtYXRlcmlhbGl6ZSB0aGVtIG5vdzogZmluaXNoX2hlYXBfc3dhcCgp
IHdpbGwgc3Vic2VxdWVudGx5IGNhbGwKKwkgKiBSZWxhdGlvbkNsZWFyTWlzc2luZygpIG9uIHRo
ZSBuZXcgcmVsYXRpb24sIGRyb3BwaW5nIHRoZSBjYXRhbG9nCisJICogbWV0YWRhdGEgdGhhdCBp
cyB0aGUgb25seSBzb3VyY2Ugb2YgdGhvc2UgdmFsdWVzLiAgV2l0aG91dCByZWZvcm1pbmcsCisJ
ICogdGhvc2UgY29sdW1ucyB3b3VsZCBzaWxlbnRseSBiZWNvbWUgTlVMTCBhZnRlciB0aGUgcmV3
cml0ZS4KKwkgKi8KKwlpZiAoSGVhcFR1cGxlSGVhZGVyR2V0TmF0dHModHVwbGUtPnRfZGF0YSkg
PCBuZXdUdXBEZXNjLT5uYXR0cykKKwkJbmVlZHNfcmVmb3JtID0gdHJ1ZTsKKwogCS8qIFNraXAg
d29yayBpZiB0aGUgdHVwbGUgZG9lc24ndCBuZWVkIGFueSBhdHRyaWJ1dGVzIGNoYW5nZWQgKi8K
LQlmb3IgKGludCBpID0gMDsgaSA8IG5ld1R1cERlc2MtPm5hdHRzOyBpKyspCisJZm9yIChpbnQg
aSA9IDA7ICFuZWVkc19yZWZvcm0gJiYgaSA8IG5ld1R1cERlc2MtPm5hdHRzOyBpKyspCiAJewog
CQlpZiAoVHVwbGVEZXNjQ29tcGFjdEF0dHIobmV3VHVwRGVzYywgaSktPmF0dGlzZHJvcHBlZCAm
JgogCQkJIWhlYXBfYXR0aXNudWxsKHR1cGxlLCBpICsgMSwgbmV3VHVwRGVzYykpCmRpZmYgLS1n
aXQgYS9zcmMvdGVzdC9yZWdyZXNzL2V4cGVjdGVkL2Zhc3RfZGVmYXVsdC5vdXQgYi9zcmMvdGVz
dC9yZWdyZXNzL2V4cGVjdGVkL2Zhc3RfZGVmYXVsdC5vdXQKaW5kZXggYmQxNDJlZDQuLjU4MTNm
MWM2IDEwMDY0NAotLS0gYS9zcmMvdGVzdC9yZWdyZXNzL2V4cGVjdGVkL2Zhc3RfZGVmYXVsdC5v
dXQKKysrIGIvc3JjL3Rlc3QvcmVncmVzcy9leHBlY3RlZC9mYXN0X2RlZmF1bHQub3V0CkBAIC05
NjksNiArOTY5LDQyIEBAIFNFTEVDVCBjb3VudCgqKQogICAgICAwCiAoMSByb3cpCiAKKy0tIFZl
cmlmeSB0aGF0IHRhYmxlLXJld3JpdGluZyBtYWludGVuYW5jZSBjb21tYW5kcyBwcmVzZXJ2ZSBh
dHRtaXNzaW5ndmFsCistLSBjb2x1bW5zLgorQ1JFQVRFIFRBQkxFIHQgKGlkIGludCBQUklNQVJZ
IEtFWSk7CitJTlNFUlQgSU5UTyB0IFNFTEVDVCBnZW5lcmF0ZV9zZXJpZXMoMSwgMyk7CitBTFRF
UiBUQUJMRSB0IEFERCBDT0xVTU4gYSBpbnQgREVGQVVMVCA0MjsKK0FMVEVSIFRBQkxFIHQgQURE
IENPTFVNTiBiIGludCBOT1QgTlVMTCBERUZBVUxUIDcgQ0hFQ0sgKGIgPiAwKTsKK1ZBQ1VVTSBG
VUxMIHQ7CitTRUxFQ1QgKiBGUk9NIHQgT1JERVIgQlkgaWQ7CisgaWQgfCBhICB8IGIgCistLS0t
Ky0tLS0rLS0tCisgIDEgfCA0MiB8IDcKKyAgMiB8IDQyIHwgNworICAzIHwgNDIgfCA3CisoMyBy
b3dzKQorCitBTFRFUiBUQUJMRSB0IEFERCBDT0xVTU4gYyB0ZXh0IERFRkFVTFQgJ2hlbGxvJzsK
K0NMVVNURVIgdCBVU0lORyB0X3BrZXk7CitTRUxFQ1QgKiBGUk9NIHQgT1JERVIgQlkgaWQ7Cisg
aWQgfCBhICB8IGIgfCAgIGMgICAKKy0tLS0rLS0tLSstLS0rLS0tLS0tLQorICAxIHwgNDIgfCA3
IHwgaGVsbG8KKyAgMiB8IDQyIHwgNyB8IGhlbGxvCisgIDMgfCA0MiB8IDcgfCBoZWxsbworKDMg
cm93cykKKworQUxURVIgVEFCTEUgdCBBREQgQ09MVU1OIGQgaW50IERFRkFVTFQgOTk7CitSRVBB
Q0sgdDsKK1NFTEVDVCAqIEZST00gdCBPUkRFUiBCWSBpZDsKKyBpZCB8IGEgIHwgYiB8ICAgYyAg
IHwgZCAgCistLS0tKy0tLS0rLS0tKy0tLS0tLS0rLS0tLQorICAxIHwgNDIgfCA3IHwgaGVsbG8g
fCA5OQorICAyIHwgNDIgfCA3IHwgaGVsbG8gfCA5OQorICAzIHwgNDIgfCA3IHwgaGVsbG8gfCA5
OQorKDMgcm93cykKKworRFJPUCBUQUJMRSB0OwogLS0gY2xlYW51cAogRFJPUCBGT1JFSUdOIFRB
QkxFIGZ0MTsKIERST1AgU0VSVkVSIHMwOwpkaWZmIC0tZ2l0IGEvc3JjL3Rlc3QvcmVncmVzcy9z
cWwvZmFzdF9kZWZhdWx0LnNxbCBiL3NyYy90ZXN0L3JlZ3Jlc3Mvc3FsL2Zhc3RfZGVmYXVsdC5z
cWwKaW5kZXggOGIzMWQzMTcuLmU1ZDlhM2QyIDEwMDY0NAotLS0gYS9zcmMvdGVzdC9yZWdyZXNz
L3NxbC9mYXN0X2RlZmF1bHQuc3FsCisrKyBiL3NyYy90ZXN0L3JlZ3Jlc3Mvc3FsL2Zhc3RfZGVm
YXVsdC5zcWwKQEAgLTY1Myw2ICs2NTMsMjIgQEAgU0VMRUNUIGNvdW50KCopCiAgIFdIRVJFIGF0
dHJlbGlkID0gJ2Z0MSc6OnJlZ2NsYXNzIEFORAogICAgIChhdHRtaXNzaW5ndmFsIElTIE5PVCBO
VUxMIE9SIGF0dGhhc21pc3NpbmcpOwogCistLSBWZXJpZnkgdGhhdCB0YWJsZS1yZXdyaXRpbmcg
bWFpbnRlbmFuY2UgY29tbWFuZHMgcHJlc2VydmUgYXR0bWlzc2luZ3ZhbAorLS0gY29sdW1ucy4K
K0NSRUFURSBUQUJMRSB0IChpZCBpbnQgUFJJTUFSWSBLRVkpOworSU5TRVJUIElOVE8gdCBTRUxF
Q1QgZ2VuZXJhdGVfc2VyaWVzKDEsIDMpOworQUxURVIgVEFCTEUgdCBBREQgQ09MVU1OIGEgaW50
IERFRkFVTFQgNDI7CitBTFRFUiBUQUJMRSB0IEFERCBDT0xVTU4gYiBpbnQgTk9UIE5VTEwgREVG
QVVMVCA3IENIRUNLIChiID4gMCk7CitWQUNVVU0gRlVMTCB0OworU0VMRUNUICogRlJPTSB0IE9S
REVSIEJZIGlkOworQUxURVIgVEFCTEUgdCBBREQgQ09MVU1OIGMgdGV4dCBERUZBVUxUICdoZWxs
byc7CitDTFVTVEVSIHQgVVNJTkcgdF9wa2V5OworU0VMRUNUICogRlJPTSB0IE9SREVSIEJZIGlk
OworQUxURVIgVEFCTEUgdCBBREQgQ09MVU1OIGQgaW50IERFRkFVTFQgOTk7CitSRVBBQ0sgdDsK
K1NFTEVDVCAqIEZST00gdCBPUkRFUiBCWSBpZDsKK0RST1AgVEFCTEUgdDsKKwogLS0gY2xlYW51
cAogRFJPUCBGT1JFSUdOIFRBQkxFIGZ0MTsKIERST1AgU0VSVkVSIHMwOwotLSAKMi40My4wCgo=
--000000000000c64b5b0650fa65bd--