public inbox for [email protected]
help / color / mirror / Atom feedsuppress empty archive_command warning message
15+ messages / 5 participants
[nested] [flat]
* suppress empty archive_command warning message
@ 2021-04-23 18:48 Pavel Tide <[email protected]>
0 siblings, 2 replies; 15+ messages in thread
From: Pavel Tide @ 2021-04-23 18:48 UTC (permalink / raw)
To: [email protected]
Hello everyone,
Is there a way to suppress the WARNING that is thrown in PG logs if
archive_mode is enabled but the archive command is empty? Maybe some
configuration key which I have overlooked?
Thanks!
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-23 19:17 Bruce Momjian <[email protected]>
parent: Pavel Tide <[email protected]>
1 sibling, 1 reply; 15+ messages in thread
From: Bruce Momjian @ 2021-04-23 19:17 UTC (permalink / raw)
To: Pavel Tide <[email protected]>; +Cc: [email protected]
On Fri, Apr 23, 2021 at 08:48:39PM +0200, Pavel Tide wrote:
> Hello everyone,
>
> Is there a way to suppress the WARNING that is thrown in PG logs if
> archive_mode is enabled but the archive command is empty? Maybe some
> configuration key which I have overlooked?
Have you tried putting "/bin/true" in there, for non-Windows?
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-23 19:18 Keith <[email protected]>
parent: Pavel Tide <[email protected]>
1 sibling, 0 replies; 15+ messages in thread
From: Keith @ 2021-04-23 19:18 UTC (permalink / raw)
To: Pavel Tide <[email protected]>; +Cc: [email protected]
On Fri, Apr 23, 2021 at 3:02 PM Pavel Tide <[email protected]> wrote:
> Hello everyone,
>
> Is there a way to suppress the WARNING that is thrown in PG logs if
> archive_mode is enabled but the archive command is empty? Maybe some
> configuration key which I have overlooked?
>
> Thanks!
>
If you're on linux, set the archive_command to "/bin/true"
Keith
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-23 19:38 Stephen Frost <[email protected]>
parent: Bruce Momjian <[email protected]>
0 siblings, 1 reply; 15+ messages in thread
From: Stephen Frost @ 2021-04-23 19:38 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Pavel Tide <[email protected]>; [email protected]
Greetings,
* Bruce Momjian ([email protected]) wrote:
> On Fri, Apr 23, 2021 at 08:48:39PM +0200, Pavel Tide wrote:
> > Is there a way to suppress the WARNING that is thrown in PG logs if
> > archive_mode is enabled but the archive command is empty? Maybe some
> > configuration key which I have overlooked?
>
> Have you tried putting "/bin/true" in there, for non-Windows?
Erm, that would cause all WAL to just be silently thrown away.. To have
the same behavior as having it be unset, you'd set it to '/bin/false'.
Of course, which you want depends, but generally you should be using a
real archive command to archive your WAL, not throwing it away.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-25 08:11 Yogesh Jadhav <[email protected]>
parent: Stephen Frost <[email protected]>
0 siblings, 2 replies; 15+ messages in thread
From: Yogesh Jadhav @ 2021-04-25 08:11 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Bruce Momjian <[email protected]>; Pavel Tide <[email protected]>; [email protected]
Greetings,
Why would anyone want to set archive_mode on and empty archive_command?
Thanks,
Yogesh
> On 24-Apr-2021, at 1:08 AM, Stephen Frost <[email protected]> wrote:
>
> Greetings,
>
> * Bruce Momjian ([email protected]) wrote:
>> On Fri, Apr 23, 2021 at 08:48:39PM +0200, Pavel Tide wrote:
>>> Is there a way to suppress the WARNING that is thrown in PG logs if
>>> archive_mode is enabled but the archive command is empty? Maybe some
>>> configuration key which I have overlooked?
>>
>> Have you tried putting "/bin/true" in there, for non-Windows?
>
> Erm, that would cause all WAL to just be silently thrown away.. To have
> the same behavior as having it be unset, you'd set it to '/bin/false'.
>
> Of course, which you want depends, but generally you should be using a
> real archive command to archive your WAL, not throwing it away.
>
> Thanks,
>
> Stephen
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-25 13:33 Pavel Tide <[email protected]>
parent: Yogesh Jadhav <[email protected]>
1 sibling, 1 reply; 15+ messages in thread
From: Pavel Tide @ 2021-04-25 13:33 UTC (permalink / raw)
To: Yogesh Jadhav <[email protected]>; +Cc: Stephen Frost <[email protected]>; Bruce Momjian <[email protected]>; [email protected]
Hi,
Thank you for your suggestions
Stephen,
Right, but /bin/false will make pgsql to retry the command again and again
and that will throw warnings as well (AFAIK).
Yogesh,
I would like to copy WALs once in every "X" minutes instead of copying them
continuously.
Cheers
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-25 17:26 Stephen Frost <[email protected]>
parent: Yogesh Jadhav <[email protected]>
1 sibling, 0 replies; 15+ messages in thread
From: Stephen Frost @ 2021-04-25 17:26 UTC (permalink / raw)
To: Yogesh Jadhav <[email protected]>; +Cc: Bruce Momjian <[email protected]>; Pavel Tide <[email protected]>; [email protected]
Greetings,
Please don't top-post on these lists.
* Yogesh Jadhav ([email protected]) wrote:
> Why would anyone want to set archive_mode on and empty archive_command?
Well, not very likely, which is why there's a warning sent out about it.
That said, when doing things like promoting a new node for a failover,
or maybe doing a restore to a new system for some kind of testing or
development work, etc, there could be reasons that you'd want archiving
enabled but not want the archive_command to actually be set (yet).
Thanks,
Stephen
> > On 24-Apr-2021, at 1:08 AM, Stephen Frost <[email protected]> wrote:
> >
> > Greetings,
> >
> > * Bruce Momjian ([email protected]) wrote:
> >> On Fri, Apr 23, 2021 at 08:48:39PM +0200, Pavel Tide wrote:
> >>> Is there a way to suppress the WARNING that is thrown in PG logs if
> >>> archive_mode is enabled but the archive command is empty? Maybe some
> >>> configuration key which I have overlooked?
> >>
> >> Have you tried putting "/bin/true" in there, for non-Windows?
> >
> > Erm, that would cause all WAL to just be silently thrown away.. To have
> > the same behavior as having it be unset, you'd set it to '/bin/false'.
> >
> > Of course, which you want depends, but generally you should be using a
> > real archive command to archive your WAL, not throwing it away.
> >
> > Thanks,
> >
> > Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-25 18:08 Stephen Frost <[email protected]>
parent: Pavel Tide <[email protected]>
0 siblings, 1 reply; 15+ messages in thread
From: Stephen Frost @ 2021-04-25 18:08 UTC (permalink / raw)
To: Pavel Tide <[email protected]>; +Cc: Yogesh Jadhav <[email protected]>; Bruce Momjian <[email protected]>; [email protected]
Greetings,
* Pavel Tide ([email protected]) wrote:
> Right, but /bin/false will make pgsql to retry the command again and again
> and that will throw warnings as well (AFAIK).
It'll say it's retrying, sure.
> I would like to copy WALs once in every "X" minutes instead of copying them
> continuously.
PG is only going to call the archice command when you've written 16MB of
WAL (unless you also have archive timeout set...).
Maybe you can clarify your use-case as to why you only want to copy them
once every 'X' minutes..?
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-26 17:29 Pavel Tide <[email protected]>
parent: Stephen Frost <[email protected]>
0 siblings, 1 reply; 15+ messages in thread
From: Pavel Tide @ 2021-04-26 17:29 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Yogesh Jadhav <[email protected]>; Bruce Momjian <[email protected]>; [email protected]
On Sun, Apr 25, 2021 at 8:08 PM Stephen Frost <[email protected]> wrote:
>
> Greetings,
>
> * Pavel Tide ([email protected]) wrote:
> > Right, but /bin/false will make pgsql to retry the command again and again
> > and that will throw warnings as well (AFAIK).
>
> It'll say it's retrying, sure.
Right, but after 3 retries (1 second between them) it will give a
warning anyway:
WARNING: archiving write-ahead log file "000000030000000500000084"
failed too many times, will try again later
Which brings us back to the initial problem.
>
> > I would like to copy WALs once in every "X" minutes instead of copying them
> > continuously.
>
> PG is only going to call the archice command when you've written 16MB of
> WAL (unless you also have archive timeout set...).
>
> Maybe you can clarify your use-case as to why you only want to copy them
> once every 'X' minutes..?
>
> Thanks,
>
> Stephen
Sure.
We have many postgres servers so we would prefer our central server to
collect logs in packs, all at once, instead of the servers streaming
them continuously.
What if we put "sleep X" insude our archive command so that it will
wait until we are ready to process the bunch of logs? Is there any
limit for how long PG will wait for the command to return something?
Thanks
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-26 17:35 Stephen Frost <[email protected]>
parent: Pavel Tide <[email protected]>
0 siblings, 1 reply; 15+ messages in thread
From: Stephen Frost @ 2021-04-26 17:35 UTC (permalink / raw)
To: Pavel Tide <[email protected]>; +Cc: Yogesh Jadhav <[email protected]>; Bruce Momjian <[email protected]>; [email protected]
Greetings,
* Pavel Tide ([email protected]) wrote:
> On Sun, Apr 25, 2021 at 8:08 PM Stephen Frost <[email protected]> wrote:
> > * Pavel Tide ([email protected]) wrote:
> > > Right, but /bin/false will make pgsql to retry the command again and again
> > > and that will throw warnings as well (AFAIK).
> >
> > It'll say it's retrying, sure.
>
> Right, but after 3 retries (1 second between them) it will give a
> warning anyway:
>
> WARNING: archiving write-ahead log file "000000030000000500000084"
> failed too many times, will try again later
>
> Which brings us back to the initial problem.
I'm not entirely sure that I agree with it being a "problem", but, yes,
that'll happen.
> > > I would like to copy WALs once in every "X" minutes instead of copying them
> > > continuously.
> >
> > PG is only going to call the archice command when you've written 16MB of
> > WAL (unless you also have archive timeout set...).
> >
> > Maybe you can clarify your use-case as to why you only want to copy them
> > once every 'X' minutes..?
[...]
> We have many postgres servers so we would prefer our central server to
> collect logs in packs, all at once, instead of the servers streaming
> them continuously.
That doesn't really provide any detail as to *why* that's sensible to
do. The longer you wait for the WAL to actually be pushed, the more
data you're potentially going to lose if something bad happens.
> What if we put "sleep X" insude our archive command so that it will
> wait until we are ready to process the bunch of logs? Is there any
> limit for how long PG will wait for the command to return something?
I hate to ask just what the archive command is that you're currently
using.. Just adding a 'sleep' is certainly not what I'd recommend as
you could end up with a pile up of WAL that's slowly trickling out after
a large amount of writes. Seems like what you'd really want is to
archive somewhere else locally and then have a cronjob or similar
periodically run and push whatever is in the WAL across to the central
server. Still don't entirely understand why this makes sense to do
though.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-26 20:34 Pavel Tide <[email protected]>
parent: Stephen Frost <[email protected]>
0 siblings, 1 reply; 15+ messages in thread
From: Pavel Tide @ 2021-04-26 20:34 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Yogesh Jadhav <[email protected]>; Bruce Momjian <[email protected]>; [email protected]
On Mon, Apr 26, 2021 at 7:35 PM Stephen Frost <[email protected]> wrote:
>
> Greetings,
>
> * Pavel Tide ([email protected]) wrote:
> > On Sun, Apr 25, 2021 at 8:08 PM Stephen Frost <[email protected]> wrote:
> > > * Pavel Tide ([email protected]) wrote:
> > > > Right, but /bin/false will make pgsql to retry the command again and again
> > > > and that will throw warnings as well (AFAIK).
> > >
> > > It'll say it's retrying, sure.
> >
> > Right, but after 3 retries (1 second between them) it will give a
> > warning anyway:
> >
> > WARNING: archiving write-ahead log file "000000030000000500000084"
> > failed too many times, will try again later
> >
> > Which brings us back to the initial problem.
>
> I'm not entirely sure that I agree with it being a "problem", but, yes,
> that'll happen.
>
> > > > I would like to copy WALs once in every "X" minutes instead of copying them
> > > > continuously.
> > >
> > > PG is only going to call the archice command when you've written 16MB of
> > > WAL (unless you also have archive timeout set...).
> > >
> > > Maybe you can clarify your use-case as to why you only want to copy them
> > > once every 'X' minutes..?
> [...]
> > We have many postgres servers so we would prefer our central server to
> > collect logs in packs, all at once, instead of the servers streaming
> > them continuously.
>
> That doesn't really provide any detail as to *why* that's sensible to
> do. The longer you wait for the WAL to actually be pushed, the more
> data you're potentially going to lose if something bad happens.
Well, our backup strategy involves having several image-based backups
and a set of corresponding WAL segments connecting those images (i1,
i2, i3). Something like this:
i1 - wal1 - wal2 - wal3 - wal4 - i2 - wal5 - wal6 - wal7 - wal8 - i3 - wal9 ...
And so on.
That way, if we wanted to restore to image2-wal7 point-in-time it
would take us less time to spin up an instance using i2 + applying
WALs 5,6,7 rather than having to restore image1 and applying WALs from
1 to 7, wouldn't it?
> > What if we put "sleep X" insude our archive command so that it will
> > wait until we are ready to process the bunch of logs? Is there any
> > limit for how long PG will wait for the command to return something?
>
> I hate to ask just what the archive command is that you're currently
> using..
It's not a command, in fact. We use an in-house-built utility that
basically places itself as an archive command once in every X minutes
and starts shipping logs to the central server.
> Just adding a 'sleep' is certainly not what I'd recommend as
> you could end up with a pile up of WAL that's slowly trickling out after
> a large amount of writes. Seems like what you'd really want is to
> archive somewhere else locally and then have a cronjob or similar
> periodically run and push whatever is in the WAL across to the central
> server.
That could be an option, but we would like to avoid any "intermediate
landing" of WAL segments.
Copying those segments onto the same machine imposes doubled IO,
whereas saving them somewhere else would require us to have some
additional storage.
Thanks
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-26 20:51 Stephen Frost <[email protected]>
parent: Pavel Tide <[email protected]>
0 siblings, 1 reply; 15+ messages in thread
From: Stephen Frost @ 2021-04-26 20:51 UTC (permalink / raw)
To: Pavel Tide <[email protected]>; +Cc: Yogesh Jadhav <[email protected]>; Bruce Momjian <[email protected]>; [email protected]
Greetings,
* Pavel Tide ([email protected]) wrote:
> On Mon, Apr 26, 2021 at 7:35 PM Stephen Frost <[email protected]> wrote:
> > * Pavel Tide ([email protected]) wrote:
> > > We have many postgres servers so we would prefer our central server to
> > > collect logs in packs, all at once, instead of the servers streaming
> > > them continuously.
> >
> > That doesn't really provide any detail as to *why* that's sensible to
> > do. The longer you wait for the WAL to actually be pushed, the more
> > data you're potentially going to lose if something bad happens.
>
> Well, our backup strategy involves having several image-based backups
> and a set of corresponding WAL segments connecting those images (i1,
> i2, i3). Something like this:
>
> i1 - wal1 - wal2 - wal3 - wal4 - i2 - wal5 - wal6 - wal7 - wal8 - i3 - wal9 ...
>
> And so on.
>
> That way, if we wanted to restore to image2-wal7 point-in-time it
> would take us less time to spin up an instance using i2 + applying
> WALs 5,6,7 rather than having to restore image1 and applying WALs from
> 1 to 7, wouldn't it?
Uh, it's not that simple though.
You absolutely *must* replay all of the WAL that existed at the time
that the snapshot was taken and only after all of that WAL has been
replayed can you stop WAL replay at some later point. There's
additional complexities if you have to deal with multiple storage
devices and tablespaces since, typically, snapshots are not guaranteed
across those and therefore you really need to actually do a
pg_start_backup and a pg_stop_backup (and save the backup label file..).
Note also that it's not obvious where exactly that ending point in the
WAL is wrt when a snapshot is taken because PG recycles WAL files and
you can't tell just by looking at the files on the filesystem which are
necessary to get back to consistency and which are from after that
point. PG also won't complain if you stop early, it'll just think that
it reached the end of WAL and will happily come up even though the
database will be corrupt.
In general, I discourage people from trying to build their own backup
system because it's simply not that easy to do correctly and you end up
with bad backups or corrupted databases when you restore...
> > > What if we put "sleep X" insude our archive command so that it will
> > > wait until we are ready to process the bunch of logs? Is there any
> > > limit for how long PG will wait for the command to return something?
> >
> > I hate to ask just what the archive command is that you're currently
> > using..
>
> It's not a command, in fact. We use an in-house-built utility that
> basically places itself as an archive command once in every X minutes
> and starts shipping logs to the central server.
What does that mean "places itself as an archive command"? You
absolutely can not just start copying WAL files out of the pg_wal
directory independently because PG recycles WAL files and the writes
into them and you don't really "know" when a WAL file has been finished
without taking other steps or arranging to have WAL files archived
through calls to archive_command...
> > Just adding a 'sleep' is certainly not what I'd recommend as
> > you could end up with a pile up of WAL that's slowly trickling out after
> > a large amount of writes. Seems like what you'd really want is to
> > archive somewhere else locally and then have a cronjob or similar
> > periodically run and push whatever is in the WAL across to the central
> > server.
>
> That could be an option, but we would like to avoid any "intermediate
> landing" of WAL segments.
> Copying those segments onto the same machine imposes doubled IO,
> whereas saving them somewhere else would require us to have some
> additional storage.
None of this explains why you want to wait to ship WAL to the central
server...
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-26 22:30 Pavel Tide <[email protected]>
parent: Stephen Frost <[email protected]>
0 siblings, 1 reply; 15+ messages in thread
From: Pavel Tide @ 2021-04-26 22:30 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Yogesh Jadhav <[email protected]>; Bruce Momjian <[email protected]>; [email protected]
> Uh, it's not that simple though.
>
> You absolutely *must* replay all of the WAL that existed at the time
> that the snapshot was taken and only after all of that WAL has been
> replayed can you stop WAL replay at some later point. There's
> additional complexities if you have to deal with multiple storage
> devices and tablespaces since, typically, snapshots are not guaranteed
> across those and therefore you really need to actually do a
> pg_start_backup and a pg_stop_backup (and save the backup label file..).
We do trigger a pg_start_backup right before taking a snapshot
(simultaneous across all devices), and once the snapshot has been
taken we trigger pg_stop_backup.
> What does that mean "places itself as an archive command"? You
> absolutely can not just start copying WAL files out of the pg_wal
> directory independently because PG recycles WAL files and the writes
> into them and you don't really "know" when a WAL file has been finished
> without taking other steps or arranging to have WAL files archived
> through calls to archive_command...
I mean that we use it as an archive_command.
Instead of placing some sort of "cp %p /mnt/nfs/%f" in
archive_command, we use '/bin/paveltide_utility %p'.
> None of this explains why you want to wait to ship WAL to the central
> server...
The server does not just accept the segments, but also uses its own
database to keep a note of the segments saved on the storage, LSNs,
and whatnot.
Continuous shipping means a higher load on the server, which we would
like to avoid.
Thanks!
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-27 14:59 Stephen Frost <[email protected]>
parent: Pavel Tide <[email protected]>
0 siblings, 1 reply; 15+ messages in thread
From: Stephen Frost @ 2021-04-27 14:59 UTC (permalink / raw)
To: Pavel Tide <[email protected]>; +Cc: Yogesh Jadhav <[email protected]>; Bruce Momjian <[email protected]>; [email protected]
Greetings,
* Pavel Tide ([email protected]) wrote:
> > You absolutely *must* replay all of the WAL that existed at the time
> > that the snapshot was taken and only after all of that WAL has been
> > replayed can you stop WAL replay at some later point. There's
> > additional complexities if you have to deal with multiple storage
> > devices and tablespaces since, typically, snapshots are not guaranteed
> > across those and therefore you really need to actually do a
> > pg_start_backup and a pg_stop_backup (and save the backup label file..).
>
> We do trigger a pg_start_backup right before taking a snapshot
> (simultaneous across all devices), and once the snapshot has been
> taken we trigger pg_stop_backup.
Ok, good, and you collect the backup_label that's returned by
pg_stop_backup and make sure to store it with that snapshot, and ensure
that when the snapshot is used you put the backup_label into place?
> > What does that mean "places itself as an archive command"? You
> > absolutely can not just start copying WAL files out of the pg_wal
> > directory independently because PG recycles WAL files and the writes
> > into them and you don't really "know" when a WAL file has been finished
> > without taking other steps or arranging to have WAL files archived
> > through calls to archive_command...
>
> I mean that we use it as an archive_command.
> Instead of placing some sort of "cp %p /mnt/nfs/%f" in
> archive_command, we use '/bin/paveltide_utility %p'.
Ok, that certainly wasn't clear from what you had written before, but if
you're at least archiving it through the archive_command then it should
be alright.
> > None of this explains why you want to wait to ship WAL to the central
> > server...
>
> The server does not just accept the segments, but also uses its own
> database to keep a note of the segments saved on the storage, LSNs,
> and whatnot.
> Continuous shipping means a higher load on the server, which we would
> like to avoid.
This doesn't make sense though- how would continuous shipping result in
a higher load? The amount of WAL doesn't change as it's based on the
amount of data written to the database and so delaying it just means
you're going to have spikes of activity and then periods of downtime.
Generally speaking, it's better to have a continuous lower level of
activity rather than such spikes (which is why we actually just bumped
the default for checkpoint completion target to 0.9, as an example...).
It's also better to get the WAL off the system as quickly as possible,
to minimize the risk of commits being lost.
The way we'd probably want to actually implement this would be some kind
of "pause/resume" system for archiving of WAL rather than just
suppressing messages about pretty clear misconfigurations, or suppress
the message about retrying failed attempts to archive a WAL segment, and
that'd probably be a fair bit of code (and a potential foot-gun for
users..) and I'm having a pretty hard time seeing the justification for
it.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: suppress empty archive_command warning message
@ 2021-04-29 17:34 Pavel Tide <[email protected]>
parent: Stephen Frost <[email protected]>
0 siblings, 0 replies; 15+ messages in thread
From: Pavel Tide @ 2021-04-29 17:34 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Yogesh Jadhav <[email protected]>; Bruce Momjian <[email protected]>; [email protected]
On Tue, Apr 27, 2021 at 4:59 PM Stephen Frost <[email protected]> wrote:
>
> Greetings,
>
> * Pavel Tide ([email protected]) wrote:
> > > You absolutely *must* replay all of the WAL that existed at the time
> > > that the snapshot was taken and only after all of that WAL has been
> > > replayed can you stop WAL replay at some later point. There's
> > > additional complexities if you have to deal with multiple storage
> > > devices and tablespaces since, typically, snapshots are not guaranteed
> > > across those and therefore you really need to actually do a
> > > pg_start_backup and a pg_stop_backup (and save the backup label file..).
> >
> > We do trigger a pg_start_backup right before taking a snapshot
> > (simultaneous across all devices), and once the snapshot has been
> > taken we trigger pg_stop_backup.
>
> Ok, good, and you collect the backup_label that's returned by
> pg_stop_backup and make sure to store it with that snapshot, and ensure
> that when the snapshot is used you put the backup_label into place?
Not that I've implemented that part by myself, but I will double-check
with my team, thank you for the advice.
> This doesn't make sense though- how would continuous shipping result in
> a higher load? The amount of WAL doesn't change as it's based on the
> amount of data written to the database and so delaying it just means
> you're going to have spikes of activity and then periods of downtime.
> Generally speaking, it's better to have a continuous lower level of
> activity rather than such spikes (which is why we actually just bumped
> the default for checkpoint completion target to 0.9, as an example...).
> It's also better to get the WAL off the system as quickly as possible,
> to minimize the risk of commits being lost.
Well, writing WAL-records in bunches instead of writing them one by
one puts less stress onto our database.
Also it might be tricky to make it so that the utility that we use
will be able to tell the server that it's ready to ship something so
that the server will come over and fetch the logs.
Therefore, the server should come and ask once in a while (depending
on how often a new segment is created, which is practically an
unpredictable thing because it depends on many factors), and that
would require the server to generate requests quite frequently.
> The way we'd probably want to actually implement this would be some kind
> of "pause/resume" system for archiving of WAL rather than just
> suppressing messages about pretty clear misconfigurations, or suppress
> the message about retrying failed attempts to archive a WAL segment, and
> that'd probably be a fair bit of code (and a potential foot-gun for
> users..) and I'm having a pretty hard time seeing the justification for
> it.
I think that pause/resume would work too.
While I agree with a foot-gun argument in general, I think that users
who subconsciously want to shoot themselves in a foot they, uh, find a
way no matter what : )
I also understand your concerns regarding the proposed design, so
thank you for your insights,
Cheers
^ permalink raw reply [nested|flat] 15+ messages in thread
end of thread, other threads:[~2021-04-29 17:34 UTC | newest]
Thread overview: 15+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-04-23 18:48 suppress empty archive_command warning message Pavel Tide <[email protected]>
2021-04-23 19:17 ` Bruce Momjian <[email protected]>
2021-04-23 19:38 ` Stephen Frost <[email protected]>
2021-04-25 08:11 ` Yogesh Jadhav <[email protected]>
2021-04-25 13:33 ` Pavel Tide <[email protected]>
2021-04-25 18:08 ` Stephen Frost <[email protected]>
2021-04-26 17:29 ` Pavel Tide <[email protected]>
2021-04-26 17:35 ` Stephen Frost <[email protected]>
2021-04-26 20:34 ` Pavel Tide <[email protected]>
2021-04-26 20:51 ` Stephen Frost <[email protected]>
2021-04-26 22:30 ` Pavel Tide <[email protected]>
2021-04-27 14:59 ` Stephen Frost <[email protected]>
2021-04-29 17:34 ` Pavel Tide <[email protected]>
2021-04-25 17:26 ` Stephen Frost <[email protected]>
2021-04-23 19:18 ` Keith <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox