pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Test that vacuum removes tuples older than OldestXmin 7+ messages / 1 participants [nested] [flat]
* pgsql: Test that vacuum removes tuples older than OldestXmin @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Test that vacuum removes tuples older than OldestXmin If vacuum fails to prune a tuple killed before OldestXmin, it will later find that tuple dead in lazy_scan_prune() and loop infinitely. Add a test reproducing this scenario to the recovery suite which creates a table on a primary, updates the table to generate dead tuples for vacuum, and then, during the vacuum, uses a replica to force GlobalVisState->maybe_needed on the primary to move backwards and precede the value of OldestXmin set at the beginning of vacuuming the table. This commit is separate from the fix in case there are test stability issues. Discussion of the bug: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com Discussion of the test: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Author: Melanie Plageman Reviewed-by: Peter Geoghegan Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/01e2b7f0fd02a44e1d13357b20fecd79919d9031 Modified Files -------------- src/test/recovery/t/043_vacuum_horizon_floor.pl | 255 ++++++++++++++++++++++++ 1 file changed, 255 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Test that vacuum removes tuples older than OldestXmin @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Test that vacuum removes tuples older than OldestXmin If vacuum fails to prune a tuple killed before OldestXmin, it will decide to freeze its xmax and later error out in pre-freeze checks. Add a test reproducing this scenario to the recovery suite which creates a table on a primary, updates the table to generate dead tuples for vacuum, and then, during the vacuum, uses a replica to force GlobalVisState->maybe_needed on the primary to move backwards and precede the value of OldestXmin set at the beginning of vacuuming the table. This commit is separate from the fix in case there are test stability issues. Author: Melanie Plageman Reviewed-by: Peter Geoghegan Discussion: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/aa607980aee08416211f003ab41aa750f5559712 Modified Files -------------- src/test/recovery/meson.build | 1 + src/test/recovery/t/043_vacuum_horizon_floor.pl | 268 ++++++++++++++++++++++++ 2 files changed, 269 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Test that vacuum removes tuples older than OldestXmin @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Test that vacuum removes tuples older than OldestXmin If vacuum fails to prune a tuple killed before OldestXmin, it will decide to freeze its xmax and later error out in pre-freeze checks. Add a test reproducing this scenario to the recovery suite which creates a table on a primary, updates the table to generate dead tuples for vacuum, and then, during the vacuum, uses a replica to force GlobalVisState->maybe_needed on the primary to move backwards and precede the value of OldestXmin set at the beginning of vacuuming the table. This commit is separate from the fix in case there are test stability issues. Author: Melanie Plageman Reviewed-by: Peter Geoghegan Discussion: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/80c34692e8e674e3b2f150f248ef2002ae2ac3a7 Modified Files -------------- src/test/recovery/meson.build | 1 + src/test/recovery/t/043_vacuum_horizon_floor.pl | 268 ++++++++++++++++++++++++ 2 files changed, 269 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Test that vacuum removes tuples older than OldestXmin @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Test that vacuum removes tuples older than OldestXmin If vacuum fails to prune a tuple killed before OldestXmin, it will later find that tuple dead in lazy_scan_prune() and loop infinitely. Add a test reproducing this scenario to the recovery suite which creates a table on a primary, updates the table to generate dead tuples for vacuum, and then, during the vacuum, uses a replica to force GlobalVisState->maybe_needed on the primary to move backwards and precede the value of OldestXmin set at the beginning of vacuuming the table. This commit is separate from the fix in case there are test stability issues. Discussion of the bug: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com Discussion of the test: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Author: Melanie Plageman Reviewed-by: Peter Geoghegan Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/e2e992820f104defeea421fd0927b763421a0396 Modified Files -------------- src/test/recovery/t/043_vacuum_horizon_floor.pl | 246 ++++++++++++++++++++++++ 1 file changed, 246 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Test that vacuum removes tuples older than OldestXmin @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Test that vacuum removes tuples older than OldestXmin If vacuum fails to prune a tuple killed before OldestXmin, it will later find that tuple dead in lazy_scan_prune() and loop infinitely. Add a test reproducing this scenario to the recovery suite which creates a table on a primary, updates the table to generate dead tuples for vacuum, and then, during the vacuum, uses a replica to force GlobalVisState->maybe_needed on the primary to move backwards and precede the value of OldestXmin set at the beginning of vacuuming the table. This commit is separate from the fix in case there are test stability issues. Discussion of the bug: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com Discussion of the test: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Author: Melanie Plageman Reviewed-by: Peter Geoghegan Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/d42f60ccf07d89c12645efa8a41c57320d01c184 Modified Files -------------- src/test/recovery/t/043_vacuum_horizon_floor.pl | 251 ++++++++++++++++++++++++ 1 file changed, 251 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Test that vacuum removes tuples older than OldestXmin @ 2025-06-24 13:29 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Melanie Plageman @ 2025-06-24 13:29 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Test that vacuum removes tuples older than OldestXmin If vacuum fails to prune a tuple killed before OldestXmin, it will decide to freeze its xmax and later error out in pre-freeze checks. Add a test reproducing this scenario to the recovery suite which creates a table on a primary, updates the table to generate dead tuples for vacuum, and then, during the vacuum, uses a replica to force GlobalVisState->maybe_needed on the primary to move backwards and precede the value of OldestXmin set at the beginning of vacuuming the table. This test is coverage for a case fixed in 83c39a1f7f3. The test was originally committed to master in aa607980aee but later reverted in efcbb76efe4 due to test instability. The test requires multiple index passes. In Postgres 17+, vacuum uses a TID store for the dead TIDs that is very space efficient. With the old minimum maintenance_work_mem of 1 MB, it required a large number of dead rows to generate enough dead TIDs to force multiple index vacuuming passes. Once the source code changes were made to allow a minimum maintenance_work_mem value of 64kB, the test could be made much faster and more stable. Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: John Naylor <johncnaylorls@gmail.com> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://postgr.es/m/CAAKRu_ZJBkidusDut6i%3DbDCiXzJEp93GC1%2BNFaZt4eqanYF3Kw%40mail.gmail.com Backpatch-through: 17 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2c0bc4765741e2947b2bd651077852935cfecb41 Modified Files -------------- src/test/recovery/meson.build | 3 +- src/test/recovery/t/048_vacuum_horizon_floor.pl | 278 ++++++++++++++++++++++++ 2 files changed, 280 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Test that vacuum removes tuples older than OldestXmin @ 2025-06-24 13:29 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Melanie Plageman @ 2025-06-24 13:29 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Test that vacuum removes tuples older than OldestXmin If vacuum fails to prune a tuple killed before OldestXmin, it will decide to freeze its xmax and later error out in pre-freeze checks. Add a test reproducing this scenario to the recovery suite which creates a table on a primary, updates the table to generate dead tuples for vacuum, and then, during the vacuum, uses a replica to force GlobalVisState->maybe_needed on the primary to move backwards and precede the value of OldestXmin set at the beginning of vacuuming the table. This test is coverage for a case fixed in 83c39a1f7f3. The test was originally committed to master in aa607980aee but later reverted in efcbb76efe4 due to test instability. The test requires multiple index passes. In Postgres 17+, vacuum uses a TID store for the dead TIDs that is very space efficient. With the old minimum maintenance_work_mem of 1 MB, it required a large number of dead rows to generate enough dead TIDs to force multiple index vacuuming passes. Once the source code changes were made to allow a minimum maintenance_work_mem value of 64kB, the test could be made much faster and more stable. Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: John Naylor <johncnaylorls@gmail.com> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://postgr.es/m/CAAKRu_ZJBkidusDut6i%3DbDCiXzJEp93GC1%2BNFaZt4eqanYF3Kw%40mail.gmail.com Backpatch-through: 17 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/303ba0573ce656b98620133cd17418dcd217318f Modified Files -------------- src/test/recovery/meson.build | 3 +- src/test/recovery/t/048_vacuum_horizon_floor.pl | 278 ++++++++++++++++++++++++ 2 files changed, 280 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2025-06-24 13:29 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-07-19 16:15 pgsql: Test that vacuum removes tuples older than OldestXmin Melanie Plageman <melanieplageman@gmail.com> 2024-07-19 16:15 pgsql: Test that vacuum removes tuples older than OldestXmin Melanie Plageman <melanieplageman@gmail.com> 2024-07-19 16:15 pgsql: Test that vacuum removes tuples older than OldestXmin Melanie Plageman <melanieplageman@gmail.com> 2024-07-19 16:15 pgsql: Test that vacuum removes tuples older than OldestXmin Melanie Plageman <melanieplageman@gmail.com> 2024-07-19 16:15 pgsql: Test that vacuum removes tuples older than OldestXmin Melanie Plageman <melanieplageman@gmail.com> 2025-06-24 13:29 pgsql: Test that vacuum removes tuples older than OldestXmin Melanie Plageman <melanieplageman@gmail.com> 2025-06-24 13:29 pgsql: Test that vacuum removes tuples older than OldestXmin Melanie Plageman <melanieplageman@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox