Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

public inbox for [email protected]  
help / color / mirror / Atom feed

From: Stephen Frost <[email protected]>
To: Bharath Rupireddy <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: SATYANARAYANA NARLAPURAM <[email protected]>
Cc: Euler Taveira <[email protected]>
Subject: Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)
Date: Mon, 18 Apr 2022 10:11:05 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <CALj2ACV9KTXDO=Vjf4K6v-b58x3XcyWDDzN5su2EK0gVWEPAug@mail.gmail.com>
References: <CALj2ACWrryWEbzn3gU9yqyarY5ptsUd8wrLwNgfv+Xn4tepRJQ@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<CALj2ACXfx48SvnPRS_P3OrXsjxgCXXE4u4ROF3pq3r-mkdz61g@mail.gmail.com>
	<CALj2ACV9KTXDO=Vjf4K6v-b58x3XcyWDDzN5su2EK0gVWEPAug@mail.gmail.com>

Greetings,

* Bharath Rupireddy ([email protected]) wrote:
> Thanks for the comments. Here's a new tool called pg_walcleaner which
> basically deletes (optionally archiving before deletion) the unneeded
> WAL files.
> 
> Please provide your thoughts and review the patches.

Alright, I spent some more time thinking about this and contemplating
what the next steps are... and I feel like the next step is basically
"add a HINT when the server can't start due to being out of disk space
that one should consider running pg_walcleaner" and at that point... why
aren't we just, uh, doing that?  This is all still quite hand-wavy, but
it sure would be nice to be able to avoid downtime due to a broken
archiving setup.  pgbackrest has a way of doing this and while we, of
course, discourage the use of that option, as it means throwing away
WAL, it's an option that users have.  PG could have a similar option.
Basically, to archive_command/library what max_slot_wal_keep_size is for
slots.

That isn't to say that we shouldn't also have a tool like this, but it
generally feels like we're taking a reactive approach here rather than a
proactive one to addressing the root issue.

Thanks,

Stephen

Attachments:

  [application/pgp-signature] signature.asc (819B, ../[email protected]/2-signature.asc)
  download

view thread (2+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox