Re: Track Oldest Initialized WAL Buffer Page

public inbox for [email protected]  
help / color / mirror / Atom feed

From: Bharath Rupireddy <[email protected]>
To: Heikki Linnakangas <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: Nathan Bossart <[email protected]>
Subject: Re: Track Oldest Initialized WAL Buffer Page
Date: Wed, 18 Oct 2023 00:16:05 +0530
Message-ID: <CALj2ACVFSirOFiABrNVAA6JtPHvA9iu+wp=qkM9pdLZ5mwLaFg@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <CALj2ACVgi6LirgLDZh=FdfdvGvKAD==WTOSWcQy=AtNgPDVnKw@mail.gmail.com>
	<[email protected]>

On Mon, Jul 3, 2023 at 6:57 PM Heikki Linnakangas <[email protected]> wrote:
>

Thanks a lot for responding. Sorry for being late.

> On 07/02/2023 16:00, Bharath Rupireddy wrote:
> > Hi,
> >
> > While working on [1], I was looking for a quick way to tell if a WAL
> > record is present in the WAL buffers array without scanning but I
> > couldn't find one.
>
> /* The end-ptr of the page that contains the record */
> expectedEndPtr += XLOG_BLCKSZ - recptr % XLOG_BLCKSZ;
>
> /* get the buffer where the record is, if it's in WAL buffers at all */
> idx = XLogRecPtrToBufIdx(recptr);
>
> /* prevent the WAL buffer from being evicted while we look at it */
> LWLockAcquire(WALBufMappingLock, LW_SHARED);
>
> /* Check if the page we're interested in is in the buffer */
> found = XLogCtl->xlblocks[idx] == expectedEndPtr;
>
> LWLockRelease(WALBufMappingLock, LW_SHARED);

This is exactly what I'm doing in the 0001 patch here
https://www.postgresql.org/message-id/[email protected]....

My bad! I should have mentioned the requirement properly - I want to
avoid taking WALBufMappingLock to peek into wal_buffers to determine
if the WAL buffer page containing the required WAL record exists.

> You actually hint at the above solution here, so I'm confused. If you're
> OK with slightly stale results, you can skip the WALBufferMappingLock
> above too, and perform an atomic read of xlblocks[idx] instead.

I get that and I see GetXLogBuffer first reading xlblocks without lock
and then to confirm it anyways takes the lock again in
AdvanceXLInsertBuffer.

     * However, we don't hold a lock while we read the value. If someone has
     * just initialized the page, it's possible that we get a "torn read" of
     * the XLogRecPtr if 64-bit fetches are not atomic on this platform. In
     * that case we will see a bogus value. That's ok, we'll grab the mapping
     * lock (in AdvanceXLInsertBuffer) and retry if we see anything else than
     * the page we're looking for. But it means that when we do this unlocked
     * read, we might see a value that appears to be ahead of the page we're
     * looking for. Don't PANIC on that, until we've verified the value while
     * holding the lock.
     */

The the 0001 patch at
https://www.postgresql.org/message-id/[email protected]...
reads the WAL buffer page with WALBufMappingLock. So, the patch can
avoid WALBufMappingLock and do something like [1]:

[1]
{
    idx = XLogRecPtrToBufIdx(ptr);
    expectedEndPtr = ptr;
    expectedEndPtr += XLOG_BLCKSZ - ptr % XLOG_BLCKSZ;

    /*
     * Do a stale read of xlblocks without WALBufMappingLock. All the callers
     * of this function are expected to read WAL that's already flushed to disk
     * from WAL buffers. If this stale read says the requested WAL buffer page
     * doesn't exist, it means that the WAL buffer page either is being or has
     * already been replaced for reuse. If this stale read says the requested
     * WAL buffer page exists, we then take WALBufMappingLock and re-read the
     * xlblocks to ensure the WAL buffer page really exists and nobody is
     * replacing it meanwhile.
     */
    endptr = XLogCtl->xlblocks[idx];

    /* Requested WAL isn't available in WAL buffers. */
    if (expectedEndPtr != endptr)
        break;

    /*
     * Requested WAL is available in WAL buffers, so recheck the existence
     * under the WALBufMappingLock and read if the page still exists, otherwise
     * return.
     */
    LWLockAcquire(WALBufMappingLock, LW_SHARED);

    endptr = XLogCtl->xlblocks[idx];

    /* Requested WAL isn't available in WAL buffers. */
    if (expectedEndPtr != endptr)
        break;

    /*
     * We found the WAL buffer page containing the given XLogRecPtr.
Get starting
     * address of the page and a pointer to the right location given
     * XLogRecPtr in that page.
     */
    page = XLogCtl->pages + idx * (Size) XLOG_BLCKSZ;
    data = page + ptr % XLOG_BLCKSZ;

    return data;
}

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

view thread (3+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Track Oldest Initialized WAL Buffer Page
  In-Reply-To: <CALj2ACVFSirOFiABrNVAA6JtPHvA9iu+wp=qkM9pdLZ5mwLaFg@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox