pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Thomas Munro <tmunro@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix recovery_prefetch with low maintenance_io_concurrency.
Date: Thu, 08 Sep 2022 09:45:39 +0000
Message-ID: <E1oWE6N-001yoY-Bp@gemulon.postgresql.org> (raw)

Fix recovery_prefetch with low maintenance_io_concurrency.

We should process completed IOs *before* trying to start more, so that
it is always possible to decode one more record when the decoded record
queue is empty, even if maintenance_io_concurrency is set so low that a
single earlier WAL record might have saturated the IO queue.

That bug was hidden because the effect of maintenance_io_concurrency was
arbitrarily clamped to be at least 2.  Fix the ordering, and also remove
that clamp.  We need a special case for 0, which is now treated the same
as recovery_prefetch=off, but otherwise the number is used directly.
This allows for testing with 1, which would have made the problem
obvious in simple test scenarios.

Also add an explicit error message for missing contrecords.  It was a
bit strange that we didn't report an error already, and became a latent
bug with prefetching, since the internal state that tracks aborted
contrecords would not survive retrying, as revealed by
026_overwrite_contrecord.pl with this adjustment.  Reporting an error
prevents that.

Back-patch to 15.

Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20220831140128.GS31833%40telsasoft.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/adb466150b44d1eaf43a2d22f58ff4c545a0ed3f

Modified Files
--------------
src/backend/access/transam/xlogprefetcher.c | 54 +++++++++++++++++++----------
src/backend/access/transam/xlogreader.c     | 23 +++++++++---
src/include/access/xlogreader.h             |  2 +-
3 files changed, 56 insertions(+), 23 deletions(-)



view thread (2+ messages)  latest in thread

Message-ID: <E1oWE6N-001yoY-Bp@gemulon.postgresql.org>
Permalink:  ../E1oWE6N-001yoY-Bp@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1oWE6N-001yoY-Bp@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: tmunro@postgresql.org
  Subject: Re: pgsql: Fix recovery_prefetch with low maintenance_io_concurrency.
  In-Reply-To: <E1oWE6N-001yoY-Bp@gemulon.postgresql.org>
  References: <E1oWE6N-001yoY-Bp@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox