Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1um0xH-008YtJ-UF for pgsql-general@arkaria.postgresql.org; Wed, 13 Aug 2025 02:11:08 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1um0xG-00BACj-Df for pgsql-general@arkaria.postgresql.org; Wed, 13 Aug 2025 02:11:06 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1um0xG-00BACa-1V for pgsql-general@lists.postgresql.org; Wed, 13 Aug 2025 02:11:06 +0000 Received: from mail-oa1-x36.google.com ([2001:4860:4864:20::36]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1um0xD-000SIB-2w for pgsql-general@lists.postgresql.org; Wed, 13 Aug 2025 02:11:05 +0000 Received: by mail-oa1-x36.google.com with SMTP id 586e51a60fabf-2e95ab2704fso5253106fac.3 for ; Tue, 12 Aug 2025 19:11:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755051061; x=1755655861; darn=lists.postgresql.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=mw7+S5mVdz87KcdKM+414OW4l/c/TYlyn1QMu55vpx0=; b=hmP1wmhkh3aArOxX2QaQ0DgvE0+fMxbr1XXPI6Tmope4d7xcLsKuPMQ5vN1N4CPh4H bd9oNvbZBUjS0FPQr7/myaBz+6hFVdnMTp73RZ/hKdDgMtS+SCBfZyiutjOQVXpESIeJ j3+4n8AX6jBT8hUL87OcVllCb4Agy9M7NoU6ws+7bC58umrSjDTXvlAsQvnggEi2yeGW lp/njHVKJ7DgCxOV4aMM4gkkwRgN9CWh2hGd3t05qhA12JhdW72aSwHMOnaBSMR5RtKa 8JRFxmkgpZou8vWLAR4LOBnnzDpG5x+uqBNZz7oVz1ZJu6togz54PMxj8K2KViuKMzjv mxUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755051061; x=1755655861; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mw7+S5mVdz87KcdKM+414OW4l/c/TYlyn1QMu55vpx0=; b=l8C7Pl8vQpCCrYUIdm0DJuj+mVqGFgQ9wz2k6RGHCmyxxRGGklWxSb9dOxlBjVMJBy fLczHKiFTUha0OpxHWuM5hHzc7+Y9sFEzmi9JACAgZJrEW+/vaUSHu8Il3Wyi2zlVr/A I2rvIk1yXeZ+y32lLGFbRGuCCpPa66hREGad+GjYoyZXtr9LDZcSmmDKgG4kjDTLcF+M WHW+yBQrxH9u/XmwydBXgrU74oviHvLgGQSiIJKf7GopeQ7cPntF1fm8VivGq83Dgj7S sPvuFK3IjrKaphnzxPYUHT+rcDdoIWp/EfsM21MHWz2slCqG2qQMyPmmjCx/3CqSNohH EbjQ== X-Gm-Message-State: AOJu0YymgDAhf4driVzie7P2T2Dty9EvT1cgin29SPSOgyKSk1Cka5sc T4hlHGaOsiRRf1rGlCDcTZGB69XPAVuWgxG4sD+7dwrcSy6KlhxsBLxSzJcO26Spw18I78oMKiC 31Sz5KIw7M3xV+c7OG4bDTbQcqbHGnWYq6w== X-Gm-Gg: ASbGnct9MrVxflhNZfJ0RkWHAwswDl1Sy4ZldVhfVe2dm1VDWyIsN/2gC+TFooDJMnE /dVQ91Cxka4YAesSDZ5u/11HcujggaPH1FivNRAtSBZ/gdF1969or1s24bxpKTFubuxBpSGfm+H srOBwVsEc6QkP1OI2GZrX0O1s1lnbcdHbu5B16CJf+kZNAxZBi60iGTHnpqGrNvia0x+ABtAmjc 2H1jp97wAdKlimdF70= X-Google-Smtp-Source: AGHT+IGdwqKcRMbhOhsXynqRBFwWBerWWhLaeQv5Nb9DvW6Us0m8MM+I6Bo2mUJkZK8gO7hn2d7GjYh8DH1FrXXRAdM= X-Received: by 2002:a05:6808:50a3:b0:3f8:150b:f571 with SMTP id 5614622812f47-435d4211f7emr904106b6e.21.1755051061233; Tue, 12 Aug 2025 19:11:01 -0700 (PDT) MIME-Version: 1.0 References: <73f3723b-f279-43c6-884d-d12b3151ec9e@aklaver.com> In-Reply-To: From: Ron Johnson Date: Tue, 12 Aug 2025 22:10:50 -0400 X-Gm-Features: Ac12FXyiNnCu-uFfBtC7s2oHjCO2x_7yJjM9HQzpvTRABpyniTZBE_JXDtpG5cM Message-ID: Subject: Re: Questions about the continuity of WAL archiving To: "pgsql-generallists.postgresql.org" Content-Type: multipart/alternative; boundary="000000000000114f31063c35ab62" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000114f31063c35ab62 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable How often does your primary node crash, and then not recover due to WALs corruption or WALs not existing? If it's _ever_ happened, you should _fix that_ instead of rolling your own WAL archival.process. On Tue, Aug 12, 2025 at 10:05=E2=80=AFPM px shi wrote= : > Hi, Adrian > > Given that you are using a less then capable storage solution(S3) why do >> you think pushing the WAL from the standby to S3 would perform any >> better then what is happening with the primary WAL? >> > > I mean that archive_mode is set to on in primary and set to always in > standby. > This way, even if the primary crashes, the standby can still archive WAL > files that the primary did not archive. > > The solution is to use a more capable storage platform. >> > > However, I believe that even if we use a more capable storage platform, > it is still impossible to archive WAL files in real time. As long as > real-time archiving cannot be achieved, there will always be some WAL fil= es > that are not archived if the primary node crashes. > > Adrian Klaver =E4=BA=8E2025=E5=B9=B48=E6=9C= =8813=E6=97=A5=E5=91=A8=E4=B8=89 00:14=E5=86=99=E9=81=93=EF=BC=9A > >> On 8/12/25 01:24, px shi wrote: >> > >> > 1) What is the current archiving setup on the primary and why is >> > lagging? >> > >> > The archive command uses pgBackRest to archive to S3. Because it is >> > uploaded to S3, the archiving speed is slow, which has caused lagging. >> > >> > 2) Have you looked at archiving off the standby node while it is i= n >> > standby per: >> > >> > Yes, archiving on the standby node is disabled. Is it recommended to >> > share the WAL archive between the primary and standby nodes to avoid >> > interruptions in archiving? >> >> Given that you are using a less then capable storage solution(S3) why do >> you think pushing the WAL from the standby to S3 would perform any >> better then what is happening with the primary WAL? >> >> The solution is to use a more capable storage platform. >> >> > >> > Adrian Klaver > > > =E4=BA=8E2025=E5=B9=B48=E6=9C=888= =E6=97=A5=E5=91=A8=E4=BA=94 23:23=E5=86=99=E9=81=93=EF=BC=9A >> > >> >> -- >> Adrian Klaver >> adrian.klaver@aklaver.com >> > --=20 Death to , and butter sauce. Don't boil me, I'm still alive. lobster! --000000000000114f31063c35ab62 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
How often does your primary node crash,= =C2=A0and then not recover due to WALs corruption or WALs not existing?

If it's _ever_ happened, you should _= fix that_ instead of rolling=C2=A0your own WAL archival.process.

<= div class=3D"gmail_quote gmail_quote_container">
On Tue, Aug 12, 2025 at 10:05=E2=80=AFPM px shi <spxlyy123@gmail.com> wrote:
Hi, Adrian
Given that you are= using a less then capable storage solution(S3) why do
you think pushing= the WAL from the standby to S3 would perform any
better then what is ha= ppening with the primary WAL?
=C2=A0
I= mean that archive_mode is set to on in primary and set to always in standb= y.=C2=A0
This way, even if the primary crashes, the standby can s= till archive WAL files that the primary did not archive.

The solution is to use = a more capable storage platform.

=C2=A0= However, I believe that even if we use a more capable storage platform, it = is still impossible to archive WAL files in real time. As long as real-time= archiving cannot be achieved, there will always be some WAL files that are= not archived if the primary node crashes.

Adrian Klaver <adrian.klaver@aklav= er.com> =E4=BA=8E2025=E5=B9=B48=E6=9C=8813=E6=97=A5=E5=91=A8=E4=B8= =89 00:14=E5=86=99=E9=81=93=EF=BC=9A
On 8/12/25 01:24, px shi wrote:
>
>=C2=A0 =C2=A0 =C2=A01) What is the current archiving setup on the prima= ry and why is
>=C2=A0 =C2=A0 =C2=A0lagging?
>
>=C2=A0 =C2=A0The archive command uses pgBackRest to archive to S3. Beca= use it is
> uploaded to S3, the archiving speed is slow, which has caused lagging.=
>
>=C2=A0 =C2=A0 =C2=A02) Have you looked at archiving off the standby nod= e while it is in
>=C2=A0 =C2=A0 =C2=A0standby per:
>
> Yes, archiving on the standby node is disabled. Is it recommended to <= br> > share the WAL archive between the primary and standby nodes to avoid <= br> > interruptions in archiving?

Given that you are using a less then capable storage solution(S3) why do you think pushing the WAL from the standby to S3 would perform any
better then what is happening with the primary WAL?

The solution is to use a more capable storage platform.

>
> Adrian Klaver <adrian.klaver@aklaver.com
> <mailto:adrian.klaver@aklaver.com>> =E4=BA=8E2025=E5=B9=B48=E6=9C=888= =E6=97=A5=E5=91=A8=E4=BA=94 23:23=E5=86=99=E9=81=93=EF=BC=9A
>

--
Adrian Klaver
adrian.klave= r@aklaver.com


--
Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.
<Redacted> lobs= ter!
--000000000000114f31063c35ab62--