pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix recovery_prefetch with low maintenance_io_concurrency. 2+ messages / 1 participants [nested] [flat]
* pgsql: Fix recovery_prefetch with low maintenance_io_concurrency. @ 2022-09-08 09:45 Thomas Munro <tmunro@postgresql.org> 0 siblings, 0 replies; 2+ messages in thread From: Thomas Munro @ 2022-09-08 09:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix recovery_prefetch with low maintenance_io_concurrency. We should process completed IOs *before* trying to start more, so that it is always possible to decode one more record when the decoded record queue is empty, even if maintenance_io_concurrency is set so low that a single earlier WAL record might have saturated the IO queue. That bug was hidden because the effect of maintenance_io_concurrency was arbitrarily clamped to be at least 2. Fix the ordering, and also remove that clamp. We need a special case for 0, which is now treated the same as recovery_prefetch=off, but otherwise the number is used directly. This allows for testing with 1, which would have made the problem obvious in simple test scenarios. Also add an explicit error message for missing contrecords. It was a bit strange that we didn't report an error already, and became a latent bug with prefetching, since the internal state that tracks aborted contrecords would not survive retrying, as revealed by 026_overwrite_contrecord.pl with this adjustment. Reporting an error prevents that. Back-patch to 15. Reported-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/20220831140128.GS31833%40telsasoft.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/adb466150b44d1eaf43a2d22f58ff4c545a0ed3f Modified Files -------------- src/backend/access/transam/xlogprefetcher.c | 54 +++++++++++++++++++---------- src/backend/access/transam/xlogreader.c | 23 +++++++++--- src/include/access/xlogreader.h | 2 +- 3 files changed, 56 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
* pgsql: Fix recovery_prefetch with low maintenance_io_concurrency. @ 2022-09-08 09:45 Thomas Munro <tmunro@postgresql.org> 0 siblings, 0 replies; 2+ messages in thread From: Thomas Munro @ 2022-09-08 09:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix recovery_prefetch with low maintenance_io_concurrency. We should process completed IOs *before* trying to start more, so that it is always possible to decode one more record when the decoded record queue is empty, even if maintenance_io_concurrency is set so low that a single earlier WAL record might have saturated the IO queue. That bug was hidden because the effect of maintenance_io_concurrency was arbitrarily clamped to be at least 2. Fix the ordering, and also remove that clamp. We need a special case for 0, which is now treated the same as recovery_prefetch=off, but otherwise the number is used directly. This allows for testing with 1, which would have made the problem obvious in simple test scenarios. Also add an explicit error message for missing contrecords. It was a bit strange that we didn't report an error already, and became a latent bug with prefetching, since the internal state that tracks aborted contrecords would not survive retrying, as revealed by 026_overwrite_contrecord.pl with this adjustment. Reporting an error prevents that. Back-patch to 15. Reported-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/20220831140128.GS31833%40telsasoft.com Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/dd38ff28addc13594c0f9e2a62ef2ffa59230598 Modified Files -------------- src/backend/access/transam/xlogprefetcher.c | 54 +++++++++++++++++++---------- src/backend/access/transam/xlogreader.c | 23 +++++++++--- src/include/access/xlogreader.h | 2 +- 3 files changed, 56 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2022-09-08 09:45 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-09-08 09:45 pgsql: Fix recovery_prefetch with low maintenance_io_concurrency. Thomas Munro <tmunro@postgresql.org> 2022-09-08 09:45 pgsql: Fix recovery_prefetch with low maintenance_io_concurrency. Thomas Munro <tmunro@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox