pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Stephen Frost <sfrost@snowman.net>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: David Steele <david@pgmasters.net>
Cc: Andres Freund <andres@anarazel.de>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: WIP: WAL prefetch (another approach)
Date: Fri, 13 Nov 2020 10:13:45 -0500
Message-ID: <20201113151344.GU16415@tamriel.snowman.net> (raw)
In-Reply-To: <5437ce5b-d305-bd85-23ec-ddb15735d93f@enterprisedb.com>
References: <20200901131426.5lse2jlepm3uligv@development>
	<CA+hUKG+9NUUbdQqFNCz2SKBdyCCD7SQwJy-Y6_PZmCnc+h79ig@mail.gmail.com>
	<20200901141826.gbhqz664brtt4y2t@development>
	<CA+hUKGLr6nU7yb8CO7_DEppBmPCOkCUvE6HBza8himYV_s5nuQ@mail.gmail.com>
	<20200908231235.lsgat5ni35h6odqt@development>
	<CA+hUKG+2Vw3UAVNJSfz5_zhRcHUWEBDrpB7pyQ85Yroep0AKbw@mail.gmail.com>
	<CA+hUKGJyuSTC5sXTv-2FmtjfFPYFU8t8JPnhV4jdMKmuKJ4tHw@mail.gmail.com>
	<20201010112935.qrhm572v4djrjljo@development>
	<CA+hUKGJwuzYCLxCP7QpK54uHBxEDGNcnDF4CJuJnuvFDpn4h5A@mail.gmail.com>
	<5437ce5b-d305-bd85-23ec-ddb15735d93f@enterprisedb.com>

Greetings,

* Tomas Vondra (tomas.vondra@enterprisedb.com) wrote:
> On 11/13/20 3:20 AM, Thomas Munro wrote:
> > I'm not really sure what to do about achive restore scripts that
> > block.  That seems to be fundamentally incompatible with what I'm
> > doing here.
> 
> IMHO we can't do much about that, except for documenting it - if the
> prefetch can't work because of blocking restore script, someone has to
> fix/improve the script. No way around that, I'm afraid.

I'm a bit confused about what the issue here is- is the concern that a
restore_command is specified that isn't allowed to run concurrently but
this patch is intending to run more than one concurrently..?  There's
another patch that I was looking at for doing pre-fetching of WAL
segments, so if this is also doing that we should figure out which
patch we want..

I don't know that it's needed, but it feels likely that we could provide
a better result if we consider making changes to the restore_command API
(eg: have a way to say "please fetch this many segments ahead, and you
can put them in this directory with these filenames" or something).  I
would think we'd be able to continue supporting the existing API and
accept that it might not be as performant.

Thanks,

Stephen

Attachments:

  signature.asc (application/pgp-signature, 818B) download

view thread (174+ messages)  latest in thread

Message-ID: <20201113151344.GU16415@tamriel.snowman.net>
Permalink:  ../20201113151344.GU16415@tamriel.snowman.net/
Also on:    postgresql.org/message-id/20201113151344.GU16415@tamriel.snowman.net

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: sfrost@snowman.net, tomas.vondra@enterprisedb.com, thomas.munro@gmail.com, tomas.vondra@2ndquadrant.com, 9erthalion6@gmail.com, david@pgmasters.net, andres@anarazel.de, alvherre@2ndquadrant.com
  Subject: Re: WIP: WAL prefetch (another approach)
  In-Reply-To: <20201113151344.GU16415@tamriel.snowman.net>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox