public inbox for [email protected]  

help / color / mirror / Atom feed

From: Laurenz Albe <[email protected]>
To: Wells Oliver <[email protected]>
To: pgsql-admin <[email protected]>
Subject: Re: RDS restore failed due to WAL log and disk space-- any tidy fixes?
Date: Sun, 17 Nov 2024 16:41:38 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAOC+FBVyTiuOMF92pQG3QOrE9j8qFegvZLoi-3UmdhFNg94e2A@mail.gmail.com>
References: <CAOC+FBVyTiuOMF92pQG3QOrE9j8qFegvZLoi-3UmdhFNg94e2A@mail.gmail.com>

On Sat, 2024-11-16 at 16:33 -0800, Wells Oliver wrote:
> I provisioned an RDS instance with 2500GB space and began the restore of a database I know to be about 1750 GB using 16 jobs.
> 
> Unfortunately, it died very near the end when it ran out of disk space due to WAL log usage. Lots of:
> 
> 2024-11-17 00:07:09 UTC::@:[19861]:PANIC:  could not write to file "pg_wal/xlogtemp.19861": No space left on device
> 
> 
> And then kaboom.
> 
> I'm wondering what my course of action should be. Can I disable/reduce WAL during a restore?
> wal_level is set to replica, can this temporarily be set to minimal? Should I just eat the extra
> costs to add headroom for the WAL? Would using fewer jobs during a restore reduce the amount of WAL
> created?

If you are using minimal WAL logging and you restore the dump in a single transaction, you
should see way less WAL generated, because data inserted into the table in the same transaction
as the CREATE TABLE statement need not be WAL logged.

But you might more easily solve the problem by speeding up or disabling the WAL archiver,
so that PostgreSQL removes old WAL after the next checkpoint.

Yours,
Laurenz Albe

view thread (8+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: RDS restore failed due to WAL log and disk space-- any tidy fixes?
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox