pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedArchive directory
10+ messages / 6 participants
[nested] [flat]
* Archive directory
@ 2020-12-07 13:23 Yambu <hyambu@gmail.com>
0 siblings, 1 reply; 10+ messages in thread
From: Yambu @ 2020-12-07 13:23 UTC (permalink / raw)
To: Pgsql-admin <pgsql-admin@lists.postgresql.org>
Hello
Does postgres have a way of removing wal files stored in archive directory?
Or do I have to manually clean up this directory?
The directory im talking about is the one on master server configured as
archive_command= cp %p /replication/archive/%f
regards
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Archive directory
@ 2020-12-07 13:38 Ian Lawrence Barwick <barwick@gmail.com>
parent: Yambu <hyambu@gmail.com>
0 siblings, 1 reply; 10+ messages in thread
From: Ian Lawrence Barwick @ 2020-12-07 13:38 UTC (permalink / raw)
To: Yambu <hyambu@gmail.com>; +Cc: Pgsql-admin <pgsql-admin@lists.postgresql.org>
2020年12月7日(月) 22:23 Yambu <hyambu@gmail.com>:
>
> Hello
>
> Does postgres have a way of removing wal files stored in archive directory?
> Or do I have to manually clean up this directory?
>
> The directory im talking about is the one on master server configured as
>
> archive_command= cp %p /replication/archive/%f
Is "archive_cleanup_command" what you're looking for?
https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-ARCHIVE-CLEANUP-COMMAND
Regards
Ian Barwick
--
EnterpriseDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Archive directory
@ 2020-12-07 13:47 Yambu <hyambu@gmail.com>
parent: Ian Lawrence Barwick <barwick@gmail.com>
0 siblings, 1 reply; 10+ messages in thread
From: Yambu @ 2020-12-07 13:47 UTC (permalink / raw)
To: Ian Lawrence Barwick <barwick@gmail.com>; +Cc: Pgsql-admin <pgsql-admin@lists.postgresql.org>
Does the command ,archive_cleanup_command, delete the files from the master
server? i understand this config is done on the slave server
On Mon, Dec 7, 2020 at 3:38 PM Ian Lawrence Barwick <barwick@gmail.com>
wrote:
> 2020年12月7日(月) 22:23 Yambu <hyambu@gmail.com>:
> >
> > Hello
> >
> > Does postgres have a way of removing wal files stored in archive
> directory?
> > Or do I have to manually clean up this directory?
> >
> > The directory im talking about is the one on master server configured as
> >
> > archive_command= cp %p /replication/archive/%f
>
> Is "archive_cleanup_command" what you're looking for?
>
>
> https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-ARCHIVE-CLEANUP-COMMAND
>
> Regards
>
> Ian Barwick
>
>
> --
> EnterpriseDB: https://www.enterprisedb.com
>
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Archive directory
@ 2020-12-07 13:53 Scott Ribe <scott_ribe@elevated-dev.com>
parent: Yambu <hyambu@gmail.com>
0 siblings, 3 replies; 10+ messages in thread
From: Scott Ribe @ 2020-12-07 13:53 UTC (permalink / raw)
To: Yambu <hyambu@gmail.com>; +Cc: Ian Lawrence Barwick <barwick@gmail.com>; Pgsql-admin <pgsql-admin@lists.postgresql.org>
> On Dec 7, 2020, at 6:47 AM, Yambu <hyambu@gmail.com> wrote:
>
> Does the command ,archive_cleanup_command, delete the files from the master server? i understand this config is done on the slave server
The point of the archive command is to copy to the replica. If you're archiving on master, then using some script to copy to replica, then postgres has no way to know when it is safe to delete WAL files.
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Archive directory
@ 2020-12-07 14:01 Yambu <hyambu@gmail.com>
parent: Scott Ribe <scott_ribe@elevated-dev.com>
2 siblings, 1 reply; 10+ messages in thread
From: Yambu @ 2020-12-07 14:01 UTC (permalink / raw)
To: Scott Ribe <scott_ribe@elevated-dev.com>; +Cc: Ian Lawrence Barwick <barwick@gmail.com>; Pgsql-admin <pgsql-admin@lists.postgresql.org>
Ok, whats happens in the case when you have 2 slave servers?
On Mon, Dec 7, 2020 at 3:53 PM Scott Ribe <scott_ribe@elevated-dev.com>
wrote:
> > On Dec 7, 2020, at 6:47 AM, Yambu <hyambu@gmail.com> wrote:
> >
> > Does the command ,archive_cleanup_command, delete the files from the
> master server? i understand this config is done on the slave server
>
> The point of the archive command is to copy to the replica. If you're
> archiving on master, then using some script to copy to replica, then
> postgres has no way to know when it is safe to delete WAL files.
>
>
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Archive directory
@ 2020-12-07 14:02 Paul Förster <paul.foerster@gmail.com>
parent: Scott Ribe <scott_ribe@elevated-dev.com>
2 siblings, 1 reply; 10+ messages in thread
From: Paul Förster @ 2020-12-07 14:02 UTC (permalink / raw)
To: Scott Ribe <scott_ribe@elevated-dev.com>; +Cc: Yambu <hyambu@gmail.com>; Ian Lawrence Barwick <barwick@gmail.com>; Pgsql-admin <pgsql-admin@lists.postgresql.org>
Hi Scott,
> On 07. Dec, 2020, at 14:53, Scott Ribe <scott_ribe@elevated-dev.com> wrote:
>
> The point of the archive command is to copy to the replica. If you're archiving on master, then using some script to copy to replica, then postgres has no way to know when it is safe to delete WAL files.
in my understanding, this is only true if you don't use replication slots. If you use replication slots then the primary should know when a WAL file is no longer needed and abandoned or ready to be recycled.
Cheers,
Paul
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Archive directory
@ 2020-12-07 14:08 MichaelDBA <MichaelDBA@sqlexec.com>
parent: Scott Ribe <scott_ribe@elevated-dev.com>
2 siblings, 0 replies; 10+ messages in thread
From: MichaelDBA @ 2020-12-07 14:08 UTC (permalink / raw)
To: Scott Ribe <scott_ribe@elevated-dev.com>; +Cc: Yambu <hyambu@gmail.com>; Ian Lawrence Barwick <barwick@gmail.com>; Pgsql-admin <pgsql-admin@lists.postgresql.org>
hmmm, I disagree with that statement:
The point of the archive_command is to release WALs from the WAL
directory that have already been applied to the actual data files during
checkpoints, and send these archived WALs elsewhere, whether it be a
replica, another location on the master, or to a backup server like
Barman and pgbackrest. The ultimate goal is to preserve point in time
recovery.
So postgres doesn't care where you send the WALs, but that you just did
something with it so it can remove them from its own internal directory,
pg_wal/pg_xlog. So you can even do this: archive_command = '/bin/true'
which actually does nothing but lets PG know it can remove that WAL from
its WALDIR.
Regards,
Michael Vitale
Scott Ribe wrote on 12/7/2020 8:53 AM:
>> On Dec 7, 2020, at 6:47 AM, Yambu <hyambu@gmail.com> wrote:
>>
>> Does the command ,archive_cleanup_command, delete the files from the master server? i understand this config is done on the slave server
> The point of the archive command is to copy to the replica. If you're archiving on master, then using some script to copy to replica, then postgres has no way to know when it is safe to delete WAL files.
>
>
>
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Archive directory
@ 2020-12-07 14:11 Scott Ribe <scott_ribe@elevated-dev.com>
parent: Paul Förster <paul.foerster@gmail.com>
0 siblings, 1 reply; 10+ messages in thread
From: Scott Ribe @ 2020-12-07 14:11 UTC (permalink / raw)
To: Paul Förster <paul.foerster@gmail.com>; +Cc: Yambu <hyambu@gmail.com>; Ian Lawrence Barwick <barwick@gmail.com>; Pgsql-admin <pgsql-admin@lists.postgresql.org>
> On Dec 7, 2020, at 7:02 AM, Paul Förster <paul.foerster@gmail.com> wrote:
>
> in my understanding, this is only true if you don't use replication slots. If you use replication slots then the primary should know when a WAL file is no longer needed and abandoned or ready to be recycled.
I don't think OP is talking about the WAL files PG is managing; I think he's talking about WAL files copied into an archive directory.
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Archive directory
@ 2020-12-07 14:31 Paul Förster <paul.foerster@gmail.com>
parent: Scott Ribe <scott_ribe@elevated-dev.com>
0 siblings, 0 replies; 10+ messages in thread
From: Paul Förster @ 2020-12-07 14:31 UTC (permalink / raw)
To: Scott Ribe <scott_ribe@elevated-dev.com>; +Cc: Yambu <hyambu@gmail.com>; Ian Lawrence Barwick <barwick@gmail.com>; Pgsql-admin <pgsql-admin@lists.postgresql.org>
Hi Scott, hi Yambu,
> On 07. Dec, 2020, at 15:11, Scott Ribe <scott_ribe@elevated-dev.com> wrote:
>
>> On Dec 7, 2020, at 7:02 AM, Paul Förster <paul.foerster@gmail.com> wrote:
>>
>> in my understanding, this is only true if you don't use replication slots. If you use replication slots then the primary should know when a WAL file is no longer needed and abandoned or ready to be recycled.
>
> I don't think OP is talking about the WAL files PG is managing; I think he's talking about WAL files copied into an archive directory.
if those files already archived to some destination are referred to, then yes, you can do with them whatever you want. They can be removed if they are not needed for a PITR.
For example, we remove them time-based. Files older than a certain number of days are removed (cron job, note: the job only removes files in the archive destination, NOT in ${PGDATA}/pg_wal).
Cheers,
Paul
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: Archive directory
@ 2020-12-07 18:15 Jeff Janes <jeff.janes@gmail.com>
parent: Yambu <hyambu@gmail.com>
0 siblings, 0 replies; 10+ messages in thread
From: Jeff Janes @ 2020-12-07 18:15 UTC (permalink / raw)
To: Yambu <hyambu@gmail.com>; +Cc: Scott Ribe <scott_ribe@elevated-dev.com>; Ian Lawrence Barwick <barwick@gmail.com>; Pgsql-admin <pgsql-admin@lists.postgresql.org>
On Mon, Dec 7, 2020 at 9:01 AM Yambu <hyambu@gmail.com> wrote:
> Ok, whats happens in the case when you have 2 slave servers?
>
Then use streaming replication with replication slots, and don't use a WAL
archive at all. This is what replication slots were invented for.
If you are using an archive merely to support replication, then you are
using an archaic setup.
You might have a good reason for using a WAL archive, but you haven't
described what that reason is.
Cheers,
Jeff
^ permalink raw reply [nested|flat] 10+ messages in thread
end of thread, other threads:[~2020-12-07 18:15 UTC | newest]
Thread overview: 10+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-12-07 13:23 Archive directory Yambu <hyambu@gmail.com>
2020-12-07 13:38 ` Ian Lawrence Barwick <barwick@gmail.com>
2020-12-07 13:47 ` Yambu <hyambu@gmail.com>
2020-12-07 13:53 ` Scott Ribe <scott_ribe@elevated-dev.com>
2020-12-07 14:01 ` Yambu <hyambu@gmail.com>
2020-12-07 18:15 ` Jeff Janes <jeff.janes@gmail.com>
2020-12-07 14:02 ` Paul Förster <paul.foerster@gmail.com>
2020-12-07 14:11 ` Scott Ribe <scott_ribe@elevated-dev.com>
2020-12-07 14:31 ` Paul Förster <paul.foerster@gmail.com>
2020-12-07 14:08 ` MichaelDBA <MichaelDBA@sqlexec.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox