From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Stephen Frost <sfrost@snowman.net>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: David Steele <david@pgmasters.net>
Cc: Andres Freund <andres@anarazel.de>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: WIP: WAL prefetch (another approach)
Date: Wed, 7 Oct 2020 02:58:38 +0200
Message-ID: <20201007005838.lrpsbpckc5b7giwk@development> (raw)
In-Reply-To: <CA+hUKG+2Vw3UAVNJSfz5_zhRcHUWEBDrpB7pyQ85Yroep0AKbw@mail.gmail.com>
References: <20200803154659.nzf5qzuozts5dujp@development>
<CA+hUKGKfsWj+4qPfOkC4YtYtVO33yrCty1uWxBp3R1OSqWEcyA@mail.gmail.com>
<20200806104701.j3xnkczd6qelfnhj@development>
<CA+hUKGKciMMJbE-1+9oPKte977b++UU_X16kkOXTF_7QjP4+jA@mail.gmail.com>
<20200901131426.5lse2jlepm3uligv@development>
<CA+hUKG+9NUUbdQqFNCz2SKBdyCCD7SQwJy-Y6_PZmCnc+h79ig@mail.gmail.com>
<20200901141826.gbhqz664brtt4y2t@development>
<CA+hUKGLr6nU7yb8CO7_DEppBmPCOkCUvE6HBza8himYV_s5nuQ@mail.gmail.com>
<20200908231235.lsgat5ni35h6odqt@development>
<CA+hUKG+2Vw3UAVNJSfz5_zhRcHUWEBDrpB7pyQ85Yroep0AKbw@mail.gmail.com>
On Thu, Sep 24, 2020 at 11:38:45AM +1200, Thomas Munro wrote:
>On Wed, Sep 9, 2020 at 11:16 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>> OK, thanks for looking into this. I guess I'll wait for an updated patch
>> before testing this further. The storage has limited capacity so I'd
>> have to either reduce the amount of data/WAL or juggle with the WAL
>> segments somehow. Doesn't seem worth it.
>
>Here's a new WIP version that works for archive-based recovery in my tests.
>
>The main change I have been working on is that there is now just a
>single XLogReaderState, so no more double-reading and double-decoding
>of the WAL. It provides XLogReadRecord(), as before, but now you can
>also read further ahead with XLogReadAhead(). The user interface is
>much like before, except that the GUCs changed a bit. They are now:
>
> recovery_prefetch=on
> recovery_prefetch_fpw=off
> wal_decode_buffer_size=256kB
> maintenance_io_concurrency=10
>
>I recommend setting maintenance_io_concurrency and
>wal_decode_buffer_size much higher than those defaults.
>
I think you've left the original GUC (replaced by the buffer size) in
the postgresql.conf.sample file. Confused me for a bit ;-)
I've done a bit of testing and so far it seems to work with WAL archive,
so I'll do more testing and benchmarking over the next couple days.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, thomas.munro@gmail.com, sfrost@snowman.net, 9erthalion6@gmail.com, david@pgmasters.net, andres@anarazel.de, alvherre@2ndquadrant.com
Subject: Re: WIP: WAL prefetch (another approach)
In-Reply-To: <20201007005838.lrpsbpckc5b7giwk@development>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox