Message-ID: <14770c231bf27ea6d22376395ac8f02e41462ed5.camel@cybertec.at>
Subject: Re: RDS restore failed due to WAL log and disk space-- any tidy
 fixes?
From: Laurenz Albe <laurenz.albe@cybertec.at>
To: Wells Oliver <wells.oliver@gmail.com>, pgsql-admin
	 <pgsql-admin@postgresql.org>
Date: Sun, 17 Nov 2024 16:41:38 +0100
In-Reply-To: 
 <CAOC+FBVyTiuOMF92pQG3QOrE9j8qFegvZLoi-3UmdhFNg94e2A@mail.gmail.com>
References: 
	<CAOC+FBVyTiuOMF92pQG3QOrE9j8qFegvZLoi-3UmdhFNg94e2A@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.4 (3.52.4-2.fc40) 
MIME-Version: 1.0
Archived-At: 
 <https://www.postgresql.org/message-id/14770c231bf27ea6d22376395ac8f02e41462ed5.camel%40cybertec.at>
Precedence: bulk

On Sat, 2024-11-16 at 16:33 -0800, Wells Oliver wrote:
> I provisioned an RDS instance with 2500GB space and began the restore of =
a database I know to be about 1750 GB using 16 jobs.
>=20
> Unfortunately, it died very near the end when it ran out of disk space du=
e to WAL log usage. Lots of:
>=20
> 2024-11-17 00:07:09 UTC::@:[19861]:PANIC:  could not write to file "pg_wa=
l/xlogtemp.19861": No space left on device
>=20
>=20
> And then kaboom.
>=20
> I'm wondering what my course of action should be. Can I disable/reduce WA=
L during a restore?
> wal_level is set to replica, can this temporarily be set to minimal? Shou=
ld I just eat the extra
> costs to add headroom for the WAL? Would using fewer jobs during a restor=
e reduce the amount of WAL
> created?

If you are using minimal WAL logging and you restore the dump in a single t=
ransaction, you
should see way less WAL generated, because data inserted into the table in =
the same transaction
as the CREATE TABLE statement need not be WAL logged.

But you might more easily solve the problem by speeding up or disabling the=
 WAL archiver,
so that PostgreSQL removes old WAL after the next checkpoint.

Yours,
Laurenz Albe