Date: Fri, 13 Nov 2020 10:13:45 -0500
From: Stephen Frost <sfrost@snowman.net>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Thomas Munro <thomas.munro@gmail.com>,
	Tomas Vondra <tomas.vondra@2ndquadrant.com>,
	Dmitry Dolgov <9erthalion6@gmail.com>,
	David Steele <david@pgmasters.net>,
	Andres Freund <andres@anarazel.de>,
	Alvaro Herrera <alvherre@2ndquadrant.com>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: WIP: WAL prefetch (another approach)
Message-ID: <20201113151344.GU16415@tamriel.snowman.net>
References: <20200901131426.5lse2jlepm3uligv@development>
 <CA+hUKG+9NUUbdQqFNCz2SKBdyCCD7SQwJy-Y6_PZmCnc+h79ig@mail.gmail.com>
 <20200901141826.gbhqz664brtt4y2t@development>
 <CA+hUKGLr6nU7yb8CO7_DEppBmPCOkCUvE6HBza8himYV_s5nuQ@mail.gmail.com>
 <20200908231235.lsgat5ni35h6odqt@development>
 <CA+hUKG+2Vw3UAVNJSfz5_zhRcHUWEBDrpB7pyQ85Yroep0AKbw@mail.gmail.com>
 <CA+hUKGJyuSTC5sXTv-2FmtjfFPYFU8t8JPnhV4jdMKmuKJ4tHw@mail.gmail.com>
 <20201010112935.qrhm572v4djrjljo@development>
 <CA+hUKGJwuzYCLxCP7QpK54uHBxEDGNcnDF4CJuJnuvFDpn4h5A@mail.gmail.com>
 <5437ce5b-d305-bd85-23ec-ddb15735d93f@enterprisedb.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="OVUm/YaakhAp1Hve"
Content-Disposition: inline
In-Reply-To: <5437ce5b-d305-bd85-23ec-ddb15735d93f@enterprisedb.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk


--OVUm/YaakhAp1Hve
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Tomas Vondra (tomas.vondra@enterprisedb.com) wrote:
> On 11/13/20 3:20 AM, Thomas Munro wrote:
> > I'm not really sure what to do about achive restore scripts that
> > block.  That seems to be fundamentally incompatible with what I'm
> > doing here.
>=20
> IMHO we can't do much about that, except for documenting it - if the
> prefetch can't work because of blocking restore script, someone has to
> fix/improve the script. No way around that, I'm afraid.

I'm a bit confused about what the issue here is- is the concern that a
restore_command is specified that isn't allowed to run concurrently but
this patch is intending to run more than one concurrently..?  There's
another patch that I was looking at for doing pre-fetching of WAL
segments, so if this is also doing that we should figure out which
patch we want..

I don't know that it's needed, but it feels likely that we could provide
a better result if we consider making changes to the restore_command API
(eg: have a way to say "please fetch this many segments ahead, and you
can put them in this directory with these filenames" or something).  I
would think we'd be able to continue supporting the existing API and
accept that it might not be as performant.

Thanks,

Stephen

--OVUm/YaakhAp1Hve
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJfrqKoAAoJEO1sijiDR2RVjGYP+gLvmM1yvBkcTqWwO3/rrbdT
6rd+E4r+EyaUC4Fj41stBFwEs5o6KtKiZfI24AsRHpDrQma4Z0d5yX/LJRVdB0Rw
epXsPbYxOG4ix+ZMFO5kAYrss5YyZ1n/RO0dlXK5MsVyAMCoD8vdZ0GN0BG6Ytsr
e22IYLqVnqOiCBWKTbUwk2E5zszuAxXb+iW/NTtxHycsElrWr6Tmn2LGnXGoUvqg
mRHQK0xfwZ92CYaygrGFdS/5rowxbNbQ+qgiPu2j3LAY+7tNKVxh7ecHAB0IQSXy
kQHa5S3LEkiNB7ub+hrfPWfAECQQ6XBwgLgOxEPTJ+7PFVJxWGgHwz+wMe02tWWQ
bqr1lcVJVt6dTE7iuDzU85O1CheddT0Nc+jSJDhpJHBnBDnxMowgbofRtCTmykCW
QSNWwsYhP21CscePl/A+AQk/V9RgQv+aTslS6tDBbdsAU6fp7FOLd8w9UnEYA4Kx
/u15Go2q5LnuWgBUGeI/XajnbS/k6L7HJdyNT8PorUl+2IxiWGTRwaXKt3qQKBhj
7sOg8LZ193fEsauGQ8VMfKJkujGSUhJrG7mHzLepFXE/70h/pr/X4Wub4ZBU+NNt
n4IpXNb2MpfJqP9Ur3E5FNplIwE6DcLLm/daQmEaf03zc1Fq9iuJwBa7u8sK8+/B
JiIuH5r6Wa3k1xlV4LrQ
=AjgL
-----END PGP SIGNATURE-----

--OVUm/YaakhAp1Hve--