agora inbox for pgsql-committers@postgresql.orghelp / color / mirror / Atom feed
pgsql: Fix REPACK (CONCURRENTLY) for columns added without a table rewr 2+ messages / 1 participants [nested] [flat]
* pgsql: Fix REPACK (CONCURRENTLY) for columns added without a table rewr @ 2026-09-25 12:09 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 2+ messages in thread From: Álvaro Herrera @ 2026-09-25 12:09 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix REPACK (CONCURRENTLY) for columns added without a table rewrite When applying the concurrent data changes, REPACK deformed the decoded tuples with the descriptor of the transient relation. That descriptor has no "missing" values, because make_new_heap() gives the transient relation none of the defaults and constraints of the source relation. A tuple written before an ALTER TABLE ... ADD COLUMN that did not rewrite the table has fewer attributes than the descriptor, so the missing values came out as NULL, even in a column declared NOT NULL. If such a value belongs to the replica identity, the key used to find the target tuple was NULL too, so the tuple to modify cannot not be found. The scan key does not remember that a value is NULL, so a pass-by-reference key would even crash the comparison function. Such tuples will be rare, because new row versions are normally formed with the current descriptor. A BEFORE ROW UPDATE trigger returning OLD is one way to produce one. Fix by deforming the decoded tuples with the descriptor of the source relation, which does have the missing values. Tuples formed with it are still valid for the transient relation, whose attributes are a copy of the source relation ones. Commit 20d3fe9009dd took the same approach for INSERT and UPDATE in the executor. Expanding the decoded tuple with heap_expand_tuple() would be the other option, but that is the workaround of commit ba9f18abd that 20d3fe9009d got rid of, so do not bring it back. Add an isolation test. Author: Sami Imseih <samimseih.pg@gmail.com> Reported-by: Shihao Zhong <zhong950419@gmail.com> Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> Backpatch-through: 19 Discussion: https://postgr.es/m/CAN12+Y+NJwr5EqKVrHpPD=5+b_NRAOzU9fhrVDO3H46x2WRxoA@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/45da2c1d75663d7a692f967b77df42a84fdd6b4d Modified Files -------------- src/backend/commands/repack.c | 59 +++++++++++++---- src/backend/replication/pgrepack/pgrepack.c | 2 +- src/test/modules/injection_points/Makefile | 1 + .../expected/repack_missingval.out | 38 +++++++++++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/specs/repack_missingval.spec | 74 ++++++++++++++++++++++ 6 files changed, 160 insertions(+), 15 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
* pgsql: Fix REPACK (CONCURRENTLY) for columns added without a table rewr @ 2026-09-25 12:09 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 2+ messages in thread From: Álvaro Herrera @ 2026-09-25 12:09 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix REPACK (CONCURRENTLY) for columns added without a table rewrite When applying the concurrent data changes, REPACK deformed the decoded tuples with the descriptor of the transient relation. That descriptor has no "missing" values, because make_new_heap() gives the transient relation none of the defaults and constraints of the source relation. A tuple written before an ALTER TABLE ... ADD COLUMN that did not rewrite the table has fewer attributes than the descriptor, so the missing values came out as NULL, even in a column declared NOT NULL. If such a value belongs to the replica identity, the key used to find the target tuple was NULL too, so the tuple to modify cannot not be found. The scan key does not remember that a value is NULL, so a pass-by-reference key would even crash the comparison function. Such tuples will be rare, because new row versions are normally formed with the current descriptor. A BEFORE ROW UPDATE trigger returning OLD is one way to produce one. Fix by deforming the decoded tuples with the descriptor of the source relation, which does have the missing values. Tuples formed with it are still valid for the transient relation, whose attributes are a copy of the source relation ones. Commit 20d3fe9009dd took the same approach for INSERT and UPDATE in the executor. Expanding the decoded tuple with heap_expand_tuple() would be the other option, but that is the workaround of commit ba9f18abd that 20d3fe9009d got rid of, so do not bring it back. Add an isolation test. Author: Sami Imseih <samimseih.pg@gmail.com> Reported-by: Shihao Zhong <zhong950419@gmail.com> Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> Backpatch-through: 19 Discussion: https://postgr.es/m/CAN12+Y+NJwr5EqKVrHpPD=5+b_NRAOzU9fhrVDO3H46x2WRxoA@mail.gmail.com Branch ------ REL_19_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a4052efe2aad4d33be61bbec090210f3d538caf5 Modified Files -------------- src/backend/commands/repack.c | 59 +++++++++++++---- src/backend/replication/pgrepack/pgrepack.c | 2 +- src/test/modules/injection_points/Makefile | 1 + .../expected/repack_missingval.out | 38 +++++++++++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/specs/repack_missingval.spec | 74 ++++++++++++++++++++++ 6 files changed, 160 insertions(+), 15 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2026-09-25 12:09 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-09-25 12:09 pgsql: Fix REPACK (CONCURRENTLY) for columns added without a table rewr Álvaro Herrera <alvherre@kurilemu.de> 2026-09-25 12:09 pgsql: Fix REPACK (CONCURRENTLY) for columns added without a table rewr Álvaro Herrera <alvherre@kurilemu.de>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox