public inbox for [email protected]
help / color / mirror / Atom feedArchive logging not cleaning up pg_wal directory
3+ messages / 3 participants
[nested] [flat]
* Archive logging not cleaning up pg_wal directory
@ 2025-04-10 12:28 Justin Swanhart <[email protected]>
2025-04-10 12:48 ` Re: Archive logging not cleaning up pg_wal directory Laurenz Albe <[email protected]>
2025-04-10 12:48 ` Re: Archive logging not cleaning up pg_wal directory Ron Johnson <[email protected]>
0 siblings, 2 replies; 3+ messages in thread
From: Justin Swanhart @ 2025-04-10 12:28 UTC (permalink / raw)
To: [email protected]
Hi,
I have the following in my postgresql.conf for archive logging:
archive_command='test ! -f /var/lib/postgresql/prod_archive_logs/%f && cp
%p /var/lib/postgresql/prod_archive_logs/%f'
This command is properly copying the log files to my archive directory but
they remain in the `pg_wal` directory after they are copied, consuming
hundreds of gigs of space. Am I supposed to create a cron job to clean out
the `pg_wal` directory periodically (keep only 1 day of logs I guess?)
I tried to use mv:
archive_command='test ! -f /var/lib/postgresql/prod_archive_logs/%f && mv
%p /var/lib/postgresql/prod_archive_logs/%f'
When I did this the archives were moved away to the archive directory but
when I tried to set up replication the server could not find the archived
log to replicate from so I changed it to "cp" and now have this problem.
What am I missing?
Regards,
--Justin
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Archive logging not cleaning up pg_wal directory
2025-04-10 12:28 Archive logging not cleaning up pg_wal directory Justin Swanhart <[email protected]>
@ 2025-04-10 12:48 ` Laurenz Albe <[email protected]>
1 sibling, 0 replies; 3+ messages in thread
From: Laurenz Albe @ 2025-04-10 12:48 UTC (permalink / raw)
To: Justin Swanhart <[email protected]>; [email protected]
On Thu, 2025-04-10 at 08:28 -0400, Justin Swanhart wrote:
> I have the following in my postgresql.conf for archive logging:
> archive_command='test ! -f /var/lib/postgresql/prod_archive_logs/%f && cp %p /var/lib/postgresql/prod_archive_logs/%f'
>
> This command is properly copying the log files to my archive directory but they remain in the `pg_wal`
> directory after they are copied, consuming hundreds of gigs of space. Am I supposed to create a cron
> job to clean out the `pg_wal` directory periodically (keep only 1 day of logs I guess?)
>
> I tried to use mv:
> archive_command='test ! -f /var/lib/postgresql/prod_archive_logs/%f && mv %p /var/lib/postgresql/prod_archive_logs/%f'
>
> When I did this the archives were moved away to the archive directory but when I tried to set up
> replication the server could not find the archived log to replicate from so I changed it to "cp" and now have this problem.
>
> What am I missing?
Something else is blocking WAL removal:
https://www.cybertec-postgresql.com/en/why-does-my-pg_wal-keep-growing/
Since you mention replication, my money is on a stale replication slot.
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Archive logging not cleaning up pg_wal directory
2025-04-10 12:28 Archive logging not cleaning up pg_wal directory Justin Swanhart <[email protected]>
@ 2025-04-10 12:48 ` Ron Johnson <[email protected]>
1 sibling, 0 replies; 3+ messages in thread
From: Ron Johnson @ 2025-04-10 12:48 UTC (permalink / raw)
To: pgsql-generallists.postgresql.org <[email protected]>
On Thu, Apr 10, 2025 at 8:29 AM Justin Swanhart <[email protected]> wrote:
> Hi,
>
> I have the following in my postgresql.conf for archive logging:
> archive_command='test ! -f /var/lib/postgresql/prod_archive_logs/%f && cp
> %p /var/lib/postgresql/prod_archive_logs/%f'
>
> This command is properly copying the log files to my archive directory but
> they remain in the `pg_wal` directory after they are copied, consuming
> hundreds of gigs of space. Am I supposed to create a cron job to clean out
> the `pg_wal` directory periodically (keep only 1 day of logs I guess?)
>
No. Definitely not.
>
> I tried to use mv:
> archive_command='test ! -f /var/lib/postgresql/prod_archive_logs/%f && mv
> %p /var/lib/postgresql/prod_archive_logs/%f'
>
> When I did this the archives were moved away to the archive directory but
> when I tried to set up replication the server could not find the archived
> log to replicate from so I changed it to "cp" and now have this problem.
>
> What am I missing?
>
PG retains WAL files when it thinks it needs them. "Orphaned replication
slot" is the typical reason.
Maybe you tried to set up replication at some point in the past, but
then forgot to drop the slot?
--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2025-04-10 12:48 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-04-10 12:28 Archive logging not cleaning up pg_wal directory Justin Swanhart <[email protected]>
2025-04-10 12:48 ` Laurenz Albe <[email protected]>
2025-04-10 12:48 ` Ron Johnson <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox