From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Stephen Frost <sfrost@snowman.net>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: David Steele <david@pgmasters.net>
Cc: Andres Freund <andres@anarazel.de>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: WIP: WAL prefetch (another approach)
Date: Mon, 3 Aug 2020 17:46:59 +0200
Message-ID: <20200803154659.nzf5qzuozts5dujp@development> (raw)
In-Reply-To: <CA+hUKG+oXRpmc=yOM-GCQMt+Q4GPt9dD_idhPhFVf6f=_owwvA@mail.gmail.com>
References: <226b5950-7404-a51d-8dc7-53895b363a38@pgmasters.net>
<CA+hUKG+AzDLtAo1R3kxLW0nKRYd9euiBhW92OW1OCjkRU2E1pQ@mail.gmail.com>
<20200419114820.h62ryte7uwu46e2x@localhost>
<CA+hUKGJ7_niJzcCHX5JWFo-05pLbzwukQfFgQi4-h46yZh=tqA@mail.gmail.com>
<20200425191935.4p6qxzzu5bklodwk@localhost>
<20200502151423.yf52i63u232fdfrg@localhost>
<CA+hUKG+obB-e_VFa2uPVe52DvtpBQtC30vyo_PDURJQTr3SdJQ@mail.gmail.com>
<20200605223422.r76a5rpl2zg2xo34@development>
<20200606003632.GS6680@tamriel.snowman.net>
<CA+hUKG+oXRpmc=yOM-GCQMt+Q4GPt9dD_idhPhFVf6f=_owwvA@mail.gmail.com>
On Thu, Jul 02, 2020 at 03:09:29PM +1200, Thomas Munro wrote:
>On Sat, Jun 6, 2020 at 12:36 PM Stephen Frost <sfrost@snowman.net> wrote:
>> * Tomas Vondra (tomas.vondra@2ndquadrant.com) wrote:
>> > I wonder if we can collect some stats to measure how effective the
>> > prefetching actually is. Ultimately we want something like cache hit
>> > ratio, but we're only preloading into page cache, so we can't easily
>> > measure that. Perhaps we could measure I/O timings in redo, though?
>>
>> That would certainly be interesting, particularly as this optimization
>> seems likely to be useful on some platforms (eg, zfs, where the
>> filesystem block size is larger than ours..) and less on others
>> (traditional systems which have a smaller block size).
>
>I know one way to get information about cache hit ratios without the
>page cache fuzz factor: if you combine this patch with Andres's
>still-in-development AIO prototype and tell it to use direct IO, you
>get the undiluted truth about hits and misses by looking at the
>"prefetch" and "skip_hit" columns of the stats view. I'm hoping to
>have a bit more to say about how this patch works as a client of that
>new magic soon, but I also don't want to make this dependent on that
>(it's mostly orthogonal, apart from the "how deep is the queue" part
>which will improve with better information).
>
>FYI I am still trying to reproduce and understand the problem Tomas
>reported; more soon.
Any luck trying to reproduce thigs? Should I try again and collect some
additional debug info?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, thomas.munro@gmail.com, sfrost@snowman.net, 9erthalion6@gmail.com, david@pgmasters.net, andres@anarazel.de, alvherre@2ndquadrant.com
Subject: Re: WIP: WAL prefetch (another approach)
In-Reply-To: <20200803154659.nzf5qzuozts5dujp@development>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox