public inbox for [email protected]  

help / color / mirror / Atom feed

From: Bossart, Nathan <[email protected]>
To: Maxim Orlov <[email protected]>
To: [email protected] <[email protected]>
Subject: Re: Pre-allocating WAL files
Date: Wed, 6 Oct 2021 16:31:42 +0000
Message-ID: <[email protected]> (raw)
In-Reply-To: <163352276350.15435.3988688227048304232.pgcf@coridan.postgresql.org>
References: <[email protected]>
	<[email protected]>
	<163352276350.15435.3988688227048304232.pgcf@coridan.postgresql.org>

On 10/6/21, 5:20 AM, "Maxim Orlov" <[email protected]> wrote:
> We've looked through the code and everything looks good except few minor things:
> 1). Using dedicated bg worker seems not optimal, it introduces a lot of redundant code for little single action.
>     We'd join initial proposal of Andres to implement it as an extra fuction of checkpointer.

Thanks for taking a look!

I have been thinking about the right place to put this logic.  My
first thought is that it sounds like something that ought to go in the
WAL writer process, but as Andres noted upthread, that is undesirable
because it'll add latency for asynchronous commits.  The checkpointer
process seems to be another candidate, but I'm not totally sure how
it'll fit in.  My patch works by maintaining a small pool of pre-
allocated segments that is quickly replenished whenever one is used.
If the checkpointer is spending most of its time checkpointing, this
small pool could remain empty for long periods of time.  To keep pre-
allocating WAL while we're checkpointing, perhaps we could add another
function like CheckpointWriteDelay() that is called periodically.
There still might be several operations in CheckPointGuts() that take
a while and leave the segment pool empty, but maybe that's okay for
now.

Nathan

view thread (19+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Pre-allocating WAL files
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox