Date: Mon, 26 Apr 2021 13:35:00 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Pavel Tide <paveltide@gmail.com>
Cc: Yogesh Jadhav <pgyogesh@outlook.com>, Bruce Momjian <bruce@momjian.us>,
	pgsql-novice@lists.postgresql.org
Subject: Re: suppress empty archive_command warning message
Message-ID: <20210426173500.GX20766@tamriel.snowman.net>
References: 
 <CAAnkphW8RpmwodKyjQhT+6jaOSLG=ERd7n-9DHr0CLRSY2W+mg@mail.gmail.com>
 <20210423191750.GA7630@momjian.us>
 <20210423193809.GL20766@tamriel.snowman.net>
 <PS2PR03MB370260F10EA97D77B26FDE1AC4439@PS2PR03MB3702.apcprd03.prod.outlook.com>
 <CAAnkphU200qFbKLVOZnwccvOu8WSPUm5_32iuWTu-ZdPRQOkYw@mail.gmail.com>
 <20210425180804.GQ20766@tamriel.snowman.net>
 <CAAnkphUSwjBYfiZO2w5+AwPftuY-vWQr8Br1eaWbfuCh=caX6A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="qWvnoaMq87/3zWV3"
Content-Disposition: inline
In-Reply-To: 
 <CAAnkphUSwjBYfiZO2w5+AwPftuY-vWQr8Br1eaWbfuCh=caX6A@mail.gmail.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: 
 <https://www.postgresql.org/message-id/20210426173500.GX20766%40tamriel.snowman.net>
Precedence: bulk


--qWvnoaMq87/3zWV3
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Pavel Tide (paveltide@gmail.com) wrote:
> On Sun, Apr 25, 2021 at 8:08 PM Stephen Frost <sfrost@snowman.net> wrote:
> > * Pavel Tide (paveltide@gmail.com) wrote:
> > > Right, but /bin/false will make pgsql to retry the command again and =
again
> > > and that will throw warnings as well (AFAIK).
> >
> > It'll say it's retrying, sure.
>=20
> Right, but after 3 retries (1 second between them) it will give a
> warning anyway:
>=20
> WARNING: archiving write-ahead log file "000000030000000500000084"
> failed too many times, will try again later
>=20
> Which brings us back to the initial problem.

I'm not entirely sure that I agree with it being a "problem", but, yes,
that'll happen.

> > > I would like to copy WALs once in every "X" minutes instead of copyin=
g them
> > > continuously.
> >
> > PG is only going to call the archice command when you've written 16MB of
> > WAL (unless you also have archive timeout set...).
> >
> > Maybe you can clarify your use-case as to why you only want to copy them
> > once every 'X' minutes..?
[...]
> We have many postgres servers so we would prefer our central server to
> collect logs in packs, all at once, instead of the servers streaming
> them continuously.

That doesn't really provide any detail as to *why* that's sensible to
do.  The longer you wait for the WAL to actually be pushed, the more
data you're potentially going to lose if something bad happens.

> What if we put "sleep X" insude our archive command so that it will
> wait until we are ready to process the bunch of logs? Is there any
> limit for how long PG will wait for the command to return something?

I hate to ask just what the archive command is that you're currently
using..  Just adding a 'sleep' is certainly not what I'd recommend as
you could end up with a pile up of WAL that's slowly trickling out after
a large amount of writes.  Seems like what you'd really want is to
archive somewhere else locally and then have a cronjob or similar
periodically run and push whatever is in the WAL across to the central
server.  Still don't entirely understand why this makes sense to do
though.

Thanks,

Stephen

--qWvnoaMq87/3zWV3
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJghvnEAAoJEO1sijiDR2RVToEQAMV6YvClhqJbW++7WVVXMc36
iTQiZ2PhuWMZCgkw+wycFo7Z7lOuqSlFtlyGgX1t9n9Cl1N3W+3WB44nneggN6Ix
EoRqaJ2MtGhV4z9iMWv3/x1ToFVU9DRwgl0yZAOfJLY/8Wu/H1Krt13DJ0CYG0L+
SLtB/ThRkovzaeJvd17dkGrsRVWAVmZZ0VdPAo864TlWy7wEXk/1bpryKbmiaLdK
Nt4XFL4hYix2dFZ35CngryEGxBpKPQP+6AM0wJ/jfCYGNzZ0Nbt0e9gQW/XevDuU
nlc+H18LJdvbR1rj8iC2stn9QA0djtvnnwXjOI3jCXQl0mUG2iFMb1msAb/RgwIs
Qhnh9Xuvh3ubsxQXLv64WnFfr3gM5W7rrODrqEbxc00KjdWvNhj9XvBUY0Iv+cHj
F9eBI9O4r+xH07Q+Oac0pdM1cx/ekZB+wwDgKOglGsKco9STm5tVOf931yEyDAnL
Ugh/dWgC+VlAOZZ7XUwguhG8zHnk/nMSi0LPAQoHzGa/5m27RxvSo2YjcMCFlAED
A2Q/FyD4IHoAqqgprmq3CN2hmtS8ZTrFE59wgFUxiTMpxkN1qO3BX9xGOO0w6yZE
8NSX9LS1pOvZQW/swohdUB8Wv6Mtp9xODUiiU4otysRKRBtQpj3tP3kYv6LF229X
8PZszawuCzULK/O8iSgo
=/JnY
-----END PGP SIGNATURE-----

--qWvnoaMq87/3zWV3--