pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX 5+ messages / 1 participants [nested] [flat]
* pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 5+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Ensure vacuum removes all visibly dead tuples older than OldestXmin If vacuum fails to remove a tuple with xmax older than VacuumCutoffs->OldestXmin and younger than GlobalVisState->maybe_needed, it will loop infinitely in lazy_scan_prune(), which compares tuples' visibility information to OldestXmin. Starting in version 14, which uses GlobalVisState for visibility testing during pruning, it is possible for GlobalVisState->maybe_needed to precede OldestXmin if maybe_needed is forced to go backward while vacuum is running. This can happen if a disconnected standby with a running transaction older than VacuumCutoffs->OldestXmin reconnects to the primary after vacuum initially calculates GlobalVisState and OldestXmin. Fix this by having vacuum always remove tuples older than OldestXmin during pruning. This is okay because the standby won't replay the tuple removal until the tuple is removable. Thus, the worst that can happen is a recovery conflict. Fixes BUG# 17257 Back-patched in versions 14-17 Author: Melanie Plageman Reviewed-by: Noah Misch, Peter Geoghegan, Robert Haas, Andres Freund, and Heikki Linnakangas Discussion: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/dc6354c67017a3fe60744fcfe4936d14d72f2d7c Modified Files -------------- src/backend/access/heap/pruneheap.c | 49 +++++++++++++++++++++++++++--------- src/backend/access/heap/vacuumlazy.c | 3 ++- src/include/access/heapam.h | 1 + 3 files changed, 40 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 5+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Ensure vacuum removes all visibly dead tuples older than OldestXmin If vacuum fails to remove a tuple with xmax older than VacuumCutoffs->OldestXmin and younger than GlobalVisState->maybe_needed, it may attempt to freeze the tuple's xmax and then ERROR out in pre-freeze checks with "cannot freeze committed xmax". Fix this by having vacuum always remove tuples older than OldestXmin. It is possible for GlobalVisState->maybe_needed to precede OldestXmin if maybe_needed is forced to go backward while vacuum is running. This can happen if a disconnected standby with a running transaction older than VacuumCutoffs->OldestXmin reconnects to the primary after vacuum initially calculates GlobalVisState and OldestXmin. In back branches starting with 14, the first version using GlobalVisState, failing to remove tuples older than OldestXmin during pruning caused vacuum to infinitely loop in lazy_scan_prune(), as investigated on this [1] thread. After 1ccc1e05ae removed the retry loop in lazy_scan_prune() and stopped comparing tuples to OldestXmin, the hang could no longer happen, but we could still attempt to freeze dead tuples with xmax older than OldestXmin -- resulting in an ERROR. Fix this by always removing dead tuples with xmax older than VacuumCutoffs->OldestXmin. This is okay because the standby won't replay the tuple removal until the tuple is removable. Thus, the worst that can happen is a recovery conflict. [1] https://postgr.es/m/20240415173913.4zyyrwaftujxthf2%40awork3.anarazel.de#1b216b7768b5bd577a3d3d51bd5... Back-patch through 14 Author: Melanie Plageman Reviewed-by: Peter Geoghegan, Robert Haas, Andres Freund, Heikki Linnakangas, and Noah Misch Discussion: https://postgr.es/m/CAAKRu_bDD7oq9ZwB2OJqub5BovMG6UjEYsoK2LVttadjEqyRGg%40mail.gmail.com Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/fd4f12df5e46793a651f2d0a272f88a6cf4b358c Modified Files -------------- src/backend/access/heap/pruneheap.c | 23 ++++++++++++++++++++++- src/backend/access/heap/vacuumlazy.c | 14 +++++++------- 2 files changed, 29 insertions(+), 8 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 5+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Ensure vacuum removes all visibly dead tuples older than OldestXmin If vacuum fails to remove a tuple with xmax older than VacuumCutoffs->OldestXmin and younger than GlobalVisState->maybe_needed, it will loop infinitely in lazy_scan_prune(), which compares tuples' visibility information to OldestXmin. Starting in version 14, which uses GlobalVisState for visibility testing during pruning, it is possible for GlobalVisState->maybe_needed to precede OldestXmin if maybe_needed is forced to go backward while vacuum is running. This can happen if a disconnected standby with a running transaction older than VacuumCutoffs->OldestXmin reconnects to the primary after vacuum initially calculates GlobalVisState and OldestXmin. Fix this by having vacuum always remove tuples older than OldestXmin during pruning. This is okay because the standby won't replay the tuple removal until the tuple is removable. Thus, the worst that can happen is a recovery conflict. Fixes BUG# 17257 Back-patched in versions 14-17 Author: Melanie Plageman Reviewed-by: Noah Misch, Peter Geoghegan, Robert Haas, Andres Freund, and Heikki Linnakangas Discussion: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/45ce054c02b8ac42909b8617819ff645e5c0487f Modified Files -------------- src/backend/access/heap/pruneheap.c | 49 +++++++++++++++++++++++++++--------- src/backend/access/heap/vacuumlazy.c | 2 +- src/include/access/heapam.h | 1 + 3 files changed, 39 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 5+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Ensure vacuum removes all visibly dead tuples older than OldestXmin If vacuum fails to remove a tuple with xmax older than VacuumCutoffs->OldestXmin and younger than GlobalVisState->maybe_needed, it may attempt to freeze the tuple's xmax and then ERROR out in pre-freeze checks with "cannot freeze committed xmax". Fix this by having vacuum always remove tuples older than OldestXmin. It is possible for GlobalVisState->maybe_needed to precede OldestXmin if maybe_needed is forced to go backward while vacuum is running. This can happen if a disconnected standby with a running transaction older than VacuumCutoffs->OldestXmin reconnects to the primary after vacuum initially calculates GlobalVisState and OldestXmin. In back branches starting with 14, the first version using GlobalVisState, failing to remove tuples older than OldestXmin during pruning caused vacuum to infinitely loop in lazy_scan_prune(), as investigated on this [1] thread. After 1ccc1e05ae removed the retry loop in lazy_scan_prune() and stopped comparing tuples to OldestXmin, the hang could no longer happen, but we could still attempt to freeze dead tuples with xmax older than OldestXmin -- resulting in an ERROR. Fix this by always removing dead tuples with xmax older than VacuumCutoffs->OldestXmin. This is okay because the standby won't replay the tuple removal until the tuple is removable. Thus, the worst that can happen is a recovery conflict. [1] https://postgr.es/m/20240415173913.4zyyrwaftujxthf2%40awork3.anarazel.de#1b216b7768b5bd577a3d3d51bd5... Back-patch through 14 Author: Melanie Plageman Reviewed-by: Peter Geoghegan, Robert Haas, Andres Freund, Heikki Linnakangas, and Noah Misch Discussion: https://postgr.es/m/CAAKRu_bDD7oq9ZwB2OJqub5BovMG6UjEYsoK2LVttadjEqyRGg%40mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/83c39a1f7f3f507058d5bc9f121a42a1a2494180 Modified Files -------------- src/backend/access/heap/pruneheap.c | 23 ++++++++++++++++++++++- src/backend/access/heap/vacuumlazy.c | 14 +++++++------- 2 files changed, 29 insertions(+), 8 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX @ 2024-07-19 16:15 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 5+ messages in thread From: Melanie Plageman @ 2024-07-19 16:15 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Ensure vacuum removes all visibly dead tuples older than OldestXmin If vacuum fails to remove a tuple with xmax older than VacuumCutoffs->OldestXmin and younger than GlobalVisState->maybe_needed, it will loop infinitely in lazy_scan_prune(), which compares tuples' visibility information to OldestXmin. Starting in version 14, which uses GlobalVisState for visibility testing during pruning, it is possible for GlobalVisState->maybe_needed to precede OldestXmin if maybe_needed is forced to go backward while vacuum is running. This can happen if a disconnected standby with a running transaction older than VacuumCutoffs->OldestXmin reconnects to the primary after vacuum initially calculates GlobalVisState and OldestXmin. Fix this by having vacuum always remove tuples older than OldestXmin during pruning. This is okay because the standby won't replay the tuple removal until the tuple is removable. Thus, the worst that can happen is a recovery conflict. Fixes BUG# 17257 Back-patched in versions 14-17 Author: Melanie Plageman Reviewed-by: Noah Misch, Peter Geoghegan, Robert Haas, Andres Freund, and Heikki Linnakangas Discussion: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/06bf404cd07bd82f08a3000d5977080f945e70ca Modified Files -------------- src/backend/access/heap/pruneheap.c | 49 +++++++++++++++++++++++++++--------- src/backend/access/heap/vacuumlazy.c | 3 ++- src/include/access/heapam.h | 1 + 3 files changed, 40 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2024-07-19 16:15 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-07-19 16:15 pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX Melanie Plageman <melanieplageman@gmail.com> 2024-07-19 16:15 pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX Melanie Plageman <melanieplageman@gmail.com> 2024-07-19 16:15 pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX Melanie Plageman <melanieplageman@gmail.com> 2024-07-19 16:15 pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX Melanie Plageman <melanieplageman@gmail.com> 2024-07-19 16:15 pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX Melanie Plageman <melanieplageman@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox