public inbox for [email protected]  

help / color / mirror / Atom feed

From: Laurenz Albe <[email protected]>
To: Wells Oliver <[email protected]>
Cc: pgsql-admin <[email protected]>
Subject: Re: RDS restore failed due to WAL log and disk space-- any tidy fixes?
Date: Sun, 17 Nov 2024 19:18:58 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAOC+FBX1L6hx4DHzH34eaWyhH6NK9ywYSJMMcEJH3rHZfW86+A@mail.gmail.com>
References: <CAOC+FBVyTiuOMF92pQG3QOrE9j8qFegvZLoi-3UmdhFNg94e2A@mail.gmail.com>
	<[email protected]>
	<CAOC+FBX1L6hx4DHzH34eaWyhH6NK9ywYSJMMcEJH3rHZfW86+A@mail.gmail.com>

On Sun, 2024-11-17 at 09:12 -0800, Wells Oliver wrote:
> On Sun, Nov 17, 2024 at 7:41 AM Laurenz Albe <[email protected]> wrote:
> > On Sat, 2024-11-16 at 16:33 -0800, Wells Oliver wrote:
> > > I provisioned an RDS instance with 2500GB space and began the restore of a database I know to be about 1750 GB using 16 jobs.
> > > 
> > > Unfortunately, it died very near the end when it ran out of disk space due to WAL log usage. Lots of:
> > > 
> > > 2024-11-17 00:07:09 UTC::@:[19861]:PANIC:  could not write to file "pg_wal/xlogtemp.19861": No space left on device
> > > 
> > > 
> > > And then kaboom.
> > > 
> > > I'm wondering what my course of action should be. Can I disable/reduce WAL during a restore?
> > > wal_level is set to replica, can this temporarily be set to minimal? Should I just eat the extra
> > > costs to add headroom for the WAL? Would using fewer jobs during a restore reduce the amount of WAL
> > > created?
> > 
> > If you are using minimal WAL logging and you restore the dump in a single transaction, you
> > should see way less WAL generated, because data inserted into the table in the same transaction
> > as the CREATE TABLE statement need not be WAL logged.
> > 
> > But you might more easily solve the problem by speeding up or disabling the WAL archiver,
> > so that PostgreSQL removes old WAL after the next checkpoint.
>
> Interesting. I am migrating a pg_dump archive to a new server, in a single go. Does it make sense
> to disable (or speed up?) WAL archiving during the restore, then reenable it after the restore so
> a future replica could work? What would be the steps here? Would disabling or "speeding up" be faster?

Ah, I ignored that you were using a hosted database.  Then you probably cannot configure WAL archiving.

Yours,
Laurenz Albe

view thread (8+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: RDS restore failed due to WAL log and disk space-- any tidy fixes?
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox