MIME-Version: 1.0
References: <20201225200953.jjkrytlrzojbndh5@alap3.anarazel.de>
 <51BAAB3B-9D3C-4155-B125-42E4F06DCDF4@amazon.com> <163352276350.15435.3988688227048304232.pgcf@coridan.postgresql.org>
 <CB15BEBD-98FC-4E72-86AE-513D34014176@amazon.com> <CALT9ZEG+R9ne6xTyEVSiLhZ7wJAQ1Jq-9XVZs=zbd_jQJyPTdA@mail.gmail.com>
 <CACG=ezb5Os_ohi1caZJLKsGJc-nSSqGXFrMjmKw4qN8fG_=NEg@mail.gmail.com>
 <DF1A2298-6367-4B6E-A0BE-00E7B09489D4@amazon.com> <CALj2ACUAYU5LeYGj+EdTBULhMgbpGJTAUd9KwzbBD4ynd_jWeA@mail.gmail.com>
In-Reply-To: <CALj2ACUAYU5LeYGj+EdTBULhMgbpGJTAUd9KwzbBD4ynd_jWeA@mail.gmail.com>
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Sat, 15 Jan 2022 13:38:26 +0530
Message-ID: <CALj2ACVQMph_hH5n5eS9aYeKTXzn2TMP+E9YHoOq-+3H-DAigA@mail.gmail.com>
Subject: Re: Pre-allocating WAL files
To: "Bossart, Nathan" <bossartn@amazon.com>
Cc: Maxim Orlov <orlovmg@gmail.com>, Pavel Borisov <pashkin.elfe@gmail.com>, 
	Maxim Orlov <m.orlov@postgrespro.ru>, 
	"pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://www.postgresql.org/message-id/CALj2ACVQMph_hH5n5eS9aYeKTXzn2TMP%2BE9YHoOq-%2B3H-DAigA%40mail.gmail.com>
Precedence: bulk

On Sat, Jan 15, 2022 at 1:36 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
>
> On Thu, Jan 6, 2022 at 3:39 AM Bossart, Nathan <bossartn@amazon.com> wrote:
> >
> > On 12/30/21, 3:52 AM, "Maxim Orlov" <orlovmg@gmail.com> wrote:
> > > I did check the patch too and found it to be ok. Check and check-world are passed.
> > > Overall idea seems to be good in my opinion, but I'm not sure where is the optimal place to put the pre-allocation.
> > >
> > > On Thu, Dec 30, 2021 at 2:46 PM Pavel Borisov <pashkin.elfe@gmail.com> wrote:
> > >> I've checked the patch v7. It applies cleanly, code is good, check-world tests passed without problems.
> > >> I think it's ok to use checkpointer for this and the overall patch can be committed. But the seen performance gain makes me think again before adding this feature. I did tests myself a couple of months ago and got similar results.
> > >> Really don't know whether is it worth the effort.
> >
> > Thank you both for your review.
>
> It may have been discussed earlier, let me ask this here - IIUC the
> whole point of pre-allocating WAL files is that creating new WAL files
> of wal_segment_size requires us to write zero-filled empty pages to
> the disk which is costly. With the advent of
> fallocate/posix_fallocate, isn't file allocation going to be much
> faster on platforms where fallocate is supported? IIRC, the
> "Asynchronous and "direct" IO support for PostgreSQL." has a way to
> use fallocate. If at all, we move ahead and use fallocate, then the
> whole point of pre-allocating WAL files becomes unnecessary?
>
> Having said above, the idea of pre-allocating WAL files is still
> relevant, given the portability of fallocate/posix_fallocate.

Adding one more point: do we have any numbers like how much total time
WAL files allocation usually takes, maybe under a high-write load
server?

Regards,
Bharath Rupireddy.