pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedWAL Archive Cleanup on Master/Standby
3+ messages / 3 participants
[nested] [flat]
* WAL Archive Cleanup on Master/Standby
@ 2014-10-15 16:02 Doiron, Daniel <DoironD@advisory.com>
0 siblings, 1 reply; 3+ messages in thread
From: Doiron, Daniel @ 2014-10-15 16:02 UTC (permalink / raw)
To: pgsql-admin
I'm curious how others are cleaning up their WAL archive files and other miscellany created by the process.
Here's my setup:
Server A: archive_command executes a script that rsync's the archive file to a local folder and a matching folder on the standby server (does not mirror the folder, just pushes the file twice)
Server B: recovery.conf archive_cleanup_command uses pg_archivecleanup to clean the standby archive folder and sends log output to a cleanup.log file
What I'm left with:
- WAL archive files on the Master server that only get cleaned up if I fail over and recover using a new pg_basebackup from Server B to Server A
- .history and .backup files on the standby server
- Entries in the cleanup.log file
Right now, I'm thinking my cleanup will involve (every 6 months):
- Failing over my cluster (kidding)
- Truncating the cleanup.log file
- ?? with .history and .backup files
What are you doing?
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: WAL Archive Cleanup on Master/Standby
@ 2014-10-15 16:27 jayknowsunix@gmail.com
parent: Doiron, Daniel <DoironD@advisory.com>
0 siblings, 1 reply; 3+ messages in thread
From: jayknowsunix@gmail.com @ 2014-10-15 16:27 UTC (permalink / raw)
To: Doiron, Daniel <DoironD@advisory.com>; +Cc: pgsql-admin
Daniel,
I use a custom script that I developed as we needed a reliable copy to a pair a standby servers. That script knows which WAL segment has been successfully copied to which server, and when it knows that both copies were successful, it deletes the segment from the master. However, if one of the copies failed, the WAL segment is left on the primary where it is tried again the next time it gets called. So far, it seems fairly reliable and consistent.
--
Jay
Sent from my iPad
> On Oct 15, 2014, at 12:02 PM, "Doiron, Daniel" <DoironD@advisory.com> wrote:
>
> I’m curious how others are cleaning up their WAL archive files and other miscellany created by the process.
>
> Here’s my setup:
>
> Server A: archive_command executes a script that rsync’s the archive file to a local folder and a matching folder on the standby server (does not mirror the folder, just pushes the file twice)
>
> Server B: recovery.conf archive_cleanup_command uses pg_archivecleanup to clean the standby archive folder and sends log output to a cleanup.log file
>
> What I’m left with:
> - WAL archive files on the Master server that only get cleaned up if I fail over and recover using a new pg_basebackup from Server B to Server A
> - .history and .backup files on the standby server
> - Entries in the cleanup.log file
>
> Right now, I’m thinking my cleanup will involve (every 6 months):
> - Failing over my cluster (kidding)
> - Truncating the cleanup.log file
> - ?? with .history and .backup files
>
> What are you doing?
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: WAL Archive Cleanup on Master/Standby
@ 2014-10-15 19:57 Payal Singh <payal@omniti.com>
parent: jayknowsunix@gmail.com
0 siblings, 0 replies; 3+ messages in thread
From: Payal Singh @ 2014-10-15 19:57 UTC (permalink / raw)
To: jayknowsunix@gmail.com; +Cc: Doiron, Daniel <DoironD@advisory.com>; pgsql-admin
you can also check out omnipitr_cleanup
https://github.com/omniti-labs/omnipitr/blob/master/doc/omnipitr-cleanup.pod
Payal Singh,
Junior Database Administrator,
OmniTI Computer Consulting Inc.
Phone: 240.646.0770 x 253
On Wed, Oct 15, 2014 at 12:27 PM, <jayknowsunix@gmail.com> wrote:
> Daniel,
>
> I use a custom script that I developed as we needed a reliable copy to a
> pair a standby servers. That script knows which WAL segment has been
> successfully copied to which server, and when it knows that both copies
> were successful, it deletes the segment from the master. However, if one of
> the copies failed, the WAL segment is left on the primary where it is tried
> again the next time it gets called. So far, it seems fairly reliable and
> consistent.
> --
> Jay
>
> Sent from my iPad
>
> On Oct 15, 2014, at 12:02 PM, "Doiron, Daniel" <DoironD@advisory.com>
> wrote:
>
> I’m curious how others are cleaning up their WAL archive files and other
> miscellany created by the process.
>
>
>
> Here’s my setup:
>
>
>
> Server A: archive_command executes a script that rsync’s the archive file
> to a local folder and a matching folder on the standby server (does not
> mirror the folder, just pushes the file twice)
>
>
>
> Server B: recovery.conf archive_cleanup_command uses pg_archivecleanup to
> clean the standby archive folder and sends log output to a cleanup.log file
>
>
>
> What I’m left with:
>
> - WAL archive files on the Master server that only get cleaned
> up if I fail over and recover using a new pg_basebackup from Server B to
> Server A
>
> - .history and .backup files on the standby server
>
> - Entries in the cleanup.log file
>
>
>
> Right now, I’m thinking my cleanup will involve (every 6 months):
>
> - Failing over my cluster (kidding)
>
> - Truncating the cleanup.log file
>
> - ?? with .history and .backup files
>
>
>
> What are you doing?
>
>
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2014-10-15 19:57 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-10-15 16:02 WAL Archive Cleanup on Master/Standby Doiron, Daniel <DoironD@advisory.com>
2014-10-15 16:27 ` jayknowsunix@gmail.com
2014-10-15 19:57 ` Payal Singh <payal@omniti.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox