Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From: John Scalia <jayknowsunix@gmail.com>
Mime-Version: 1.0 (1.0)
Subject: Re: WAL restore is very slow
Date: Mon, 29 Aug 2022 06:28:55 -0400
Message-Id: <D28C7C2A-AF07-4AAE-A364-11BC57FF7DCA@gmail.com>
References: 
 <CABai+ZMELUgce33KXvMXmRcHfhRkdbmkfP6kWdq_kQC8KtU81A@mail.gmail.com>
Cc: pgsql-admin@lists.postgresql.org
In-Reply-To: 
 <CABai+ZMELUgce33KXvMXmRcHfhRkdbmkfP6kWdq_kQC8KtU81A@mail.gmail.com>
To: Madhu Sudan <madhusudan0429@gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/D28C7C2A-AF07-4AAE-A364-11BC57FF7DCA%40gmail.com>
Precedence: bulk

First off, why are you still running on a rather small r5 instance? AWS has h=
ad r6 instances available for some time, and they=E2=80=99re faster and more=
 efficient. Of my some 30 instances which I take care of, I haven=E2=80=99t h=
ad any r5=E2=80=99s in quite some time.

And it sounds here that you=E2=80=99re running on an EC2 instance as well. I=
s there some reason you haven=E2=80=99t gone to RDS? Cluster configuration i=
n RDS uses an internal RDS specific replication method that does not involve=
 WAL files.
=E2=80=94
John

Sent from my iPad

> On Aug 29, 2022, at 6:10 AM, Madhu Sudan <madhusudan0429@gmail.com> wrote:=

>=20
> =EF=BB=BF
> Hi
>=20
> We have PG-14 with a huge data set of 14 TB running on r5b.2xlarge. We hav=
e set up WAL archiving and restoring them onto a replica server. The WAL res=
tore on the replica is very slow and we are not able to achieve the 4 hour d=
elayed replica. It is always behind 30 hrs with the huge WAL generation.
>=20
> I have checked the following and they look fine
> 1. Bottlenecks on the replica server
> 2. Memory consumption and swap
> 3. EFS IO throughput
> 4.checkpoint_completion_target =3D 0.9
> 5. wal_buffers =3D 16MB
> 6. wal_log_hints =3D on
> 7. Verified logs and didn't find anything useful related to the issue
>=20
> Can you please suggest how to improve the WAL restore performance=20
>=20
> Thank you=20
> Madhu Sudan
>=20
>=20
>=20