public inbox for [email protected]  

help / color / mirror / Atom feed

From: Stephen Frost <[email protected]>
To: Pavel Tide <[email protected]>
Cc: Yogesh Jadhav <[email protected]>
Cc: Bruce Momjian <[email protected]>
Cc: [email protected]
Subject: Re: suppress empty archive_command warning message
Date: Mon, 26 Apr 2021 13:35:00 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAAnkphUSwjBYfiZO2w5+AwPftuY-vWQr8Br1eaWbfuCh=caX6A@mail.gmail.com>
References: <CAAnkphW8RpmwodKyjQhT+6jaOSLG=ERd7n-9DHr0CLRSY2W+mg@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<PS2PR03MB370260F10EA97D77B26FDE1AC4439@PS2PR03MB3702.apcprd03.prod.outlook.com>
	<CAAnkphU200qFbKLVOZnwccvOu8WSPUm5_32iuWTu-ZdPRQOkYw@mail.gmail.com>
	<[email protected]>
	<CAAnkphUSwjBYfiZO2w5+AwPftuY-vWQr8Br1eaWbfuCh=caX6A@mail.gmail.com>

Greetings,

* Pavel Tide ([email protected]) wrote:
> On Sun, Apr 25, 2021 at 8:08 PM Stephen Frost <[email protected]> wrote:
> > * Pavel Tide ([email protected]) wrote:
> > > Right, but /bin/false will make pgsql to retry the command again and again
> > > and that will throw warnings as well (AFAIK).
> >
> > It'll say it's retrying, sure.
> 
> Right, but after 3 retries (1 second between them) it will give a
> warning anyway:
> 
> WARNING: archiving write-ahead log file "000000030000000500000084"
> failed too many times, will try again later
> 
> Which brings us back to the initial problem.

I'm not entirely sure that I agree with it being a "problem", but, yes,
that'll happen.

> > > I would like to copy WALs once in every "X" minutes instead of copying them
> > > continuously.
> >
> > PG is only going to call the archice command when you've written 16MB of
> > WAL (unless you also have archive timeout set...).
> >
> > Maybe you can clarify your use-case as to why you only want to copy them
> > once every 'X' minutes..?
[...]
> We have many postgres servers so we would prefer our central server to
> collect logs in packs, all at once, instead of the servers streaming
> them continuously.

That doesn't really provide any detail as to *why* that's sensible to
do.  The longer you wait for the WAL to actually be pushed, the more
data you're potentially going to lose if something bad happens.

> What if we put "sleep X" insude our archive command so that it will
> wait until we are ready to process the bunch of logs? Is there any
> limit for how long PG will wait for the command to return something?

I hate to ask just what the archive command is that you're currently
using..  Just adding a 'sleep' is certainly not what I'd recommend as
you could end up with a pile up of WAL that's slowly trickling out after
a large amount of writes.  Seems like what you'd really want is to
archive somewhere else locally and then have a cronjob or similar
periodically run and push whatever is in the WAL across to the central
server.  Still don't entirely understand why this makes sense to do
though.

Thanks,

Stephen


Attachments:

  [application/pgp-signature] signature.asc (819B, ../[email protected]/2-signature.asc)
  download

view thread (15+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: suppress empty archive_command warning message
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox