Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1um0rh-008XpZ-5B for pgsql-general@arkaria.postgresql.org; Wed, 13 Aug 2025 02:05:21 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1um0rd-00B6Yg-4O for pgsql-general@arkaria.postgresql.org; Wed, 13 Aug 2025 02:05:17 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1um0rc-00B6YY-Jp for pgsql-general@lists.postgresql.org; Wed, 13 Aug 2025 02:05:17 +0000 Received: from mail-vk1-xa30.google.com ([2607:f8b0:4864:20::a30]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1um0ra-000Hc3-1K for pgsql-general@lists.postgresql.org; Wed, 13 Aug 2025 02:05:15 +0000 Received: by mail-vk1-xa30.google.com with SMTP id 71dfb90a1353d-5393bb30c99so909627e0c.3 for ; Tue, 12 Aug 2025 19:05:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755050713; x=1755655513; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=2qYimj1HjGzPZU4WwxeWGhvzYCEs1BczT5cSk/IUw24=; b=CPPCV/W8Nxk7fE7C3DwBhDBRLRZsPukrHx/T+X1df13E0hdkuO6OmTqYqY/9p+lMd2 KnsEDgDw3agf1OqvZL4p51NNuHiQ8mhlNl+8ZaBIymDVRKBI1uByMNW6ZJi5a73C/BHo YW6ud0yAKvMx/r3F9WlmOZCjxDEnW9be0QqMtxZLhTrK/71CRMZVBFUebRjaM+058VWB iKm01/DS7n6ci4RbNGKSuTGnrgV9LeXY6rCVQKEBAAxUmEz5ho/cs7DlA2XQZFeiEvbB vzuFq404m7C/xCizd2Y9W/1FogA3C+ME8KPJ5D9am35bJYCLmFnkrOJKTZH0XqHy356N 6Mnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755050713; x=1755655513; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2qYimj1HjGzPZU4WwxeWGhvzYCEs1BczT5cSk/IUw24=; b=AtOOiqt2w2+KhGsE3OwMD2utv2rPR0erIHbN4u75HPvcLFmI/JKxJTw5j2ncR1RBka mKYHUfJ98QAXgJGHEg7ufzE2Lq6VUiNb8GzQpX/ANMubxUUhsTYlpqepyJrBzROjWzFu EVxABqtwZQsNFu5u6U560mj+Q3alPULgUihCjOGeUol2wFsMo/keE/iMR0eMWYlg2wvp Q2wbHVJzICp8VAM1pA1UTGHTiJ9Mf36NZibdIJFm4t+ZYCG7CYbkQfFT/SeaEUkyhK5+ zi/v3DuVJwaKlBV1O2OFGF3ZQ2t/XzRkGVZSRlYPRIz3HAvCba1aZvb5Lvwdn2XBNxg2 bo7Q== X-Gm-Message-State: AOJu0Yy6HJLmQI1eQO15+p9EvGLxGzz4cpwHJzpqJgufUQUqa0QIJz/0 /4cxRE9FOFnTvCKLQeTqZDS6zM0uZ/GvfgmvZIimtFtHKsvhxBZnaK2x9ylT602c0cRKwh4xn7t LyVU86AJCYP+nlDhm2UdZqMGW/3DAelc= X-Gm-Gg: ASbGncsm6Lgu5wY8T60PDaumprfuwANlvFoRECN21toIZmyUQ7QwyK04Ch1OzX+0W8G TYu0PDkZ5yp2DC6LrqQgPC5QHcpsNvB94NLIIv/5Q/77J6vOJnLGFTWvCaOw4SJO5T1YvCDMfw6 kd+/pLZmaaER208n2TKvIfyoRDCSNwhUtGEg2phN94GB1yMPJHyFgIOTXPkzjb76oFe/j15zzmW crVCPDrRpWQ1T1LwDQrLZsq7bu5MPfoQfRG+Ip/FQ== X-Google-Smtp-Source: AGHT+IHzFK0PhiOt6tdMbhk8oQEmkApUZeQurzwKQLLNLHkkK45YTMscEx98RAGnZLNB9sCZ8z6ye85i2V/xhvv97tM= X-Received: by 2002:a05:6122:4f84:b0:534:8213:af78 with SMTP id 71dfb90a1353d-53b0b5eaccamr428980e0c.8.1755050712921; Tue, 12 Aug 2025 19:05:12 -0700 (PDT) MIME-Version: 1.0 References: <73f3723b-f279-43c6-884d-d12b3151ec9e@aklaver.com> In-Reply-To: <73f3723b-f279-43c6-884d-d12b3151ec9e@aklaver.com> From: px shi Date: Wed, 13 Aug 2025 10:05:01 +0800 X-Gm-Features: Ac12FXxY50WLRCnDmV_W9M7-WtLZj5tNAFzMRiaAkaZCtKTFhbsg8nTpKt7h-gI Message-ID: Subject: Re: Questions about the continuity of WAL archiving To: Adrian Klaver Cc: pgsql-general@lists.postgresql.org Content-Type: multipart/alternative; boundary="0000000000004e7a8d063c359656" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --0000000000004e7a8d063c359656 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, Adrian Given that you are using a less then capable storage solution(S3) why do > you think pushing the WAL from the standby to S3 would perform any > better then what is happening with the primary WAL? > I mean that archive_mode is set to on in primary and set to always in standby. This way, even if the primary crashes, the standby can still archive WAL files that the primary did not archive. The solution is to use a more capable storage platform. > However, I believe that even if we use a more capable storage platform, it is still impossible to archive WAL files in real time. As long as real-time archiving cannot be achieved, there will always be some WAL files that are not archived if the primary node crashes. Adrian Klaver =E4=BA=8E2025=E5=B9=B48=E6=9C=881= 3=E6=97=A5=E5=91=A8=E4=B8=89 00:14=E5=86=99=E9=81=93=EF=BC=9A > On 8/12/25 01:24, px shi wrote: > > > > 1) What is the current archiving setup on the primary and why is > > lagging? > > > > The archive command uses pgBackRest to archive to S3. Because it is > > uploaded to S3, the archiving speed is slow, which has caused lagging. > > > > 2) Have you looked at archiving off the standby node while it is in > > standby per: > > > > Yes, archiving on the standby node is disabled. Is it recommended to > > share the WAL archive between the primary and standby nodes to avoid > > interruptions in archiving? > > Given that you are using a less then capable storage solution(S3) why do > you think pushing the WAL from the standby to S3 would perform any > better then what is happening with the primary WAL? > > The solution is to use a more capable storage platform. > > > > > Adrian Klaver > > =E4=BA=8E2025=E5=B9=B48=E6=9C=888= =E6=97=A5=E5=91=A8=E4=BA=94 23:23=E5=86=99=E9=81=93=EF=BC=9A > > > > -- > Adrian Klaver > adrian.klaver@aklaver.com > --0000000000004e7a8d063c359656 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi, Adrian

Given that you are usi= ng a less then capable storage solution(S3) why do
you think pushing the= WAL from the standby to S3 would perform any
better then what is happen= ing with the primary WAL?
=C2=A0
I mea= n that archive_mode is set to on in primary and set to always in standby.= =C2=A0
This way, even if the primary crashes, the standby can sti= ll archive WAL files that the primary did not archive.

=
The solution is to use a more capable storage platform.

=C2=A0However, I believe that even if we = use a more capable storage platform, it is still impossible to archive WAL = files in real time. As long as real-time archiving cannot be achieved, ther= e will always be some WAL files that are not archived if the primary node c= rashes.

Adrian Klaver <adrian.klaver@aklaver.com> =E4=BA=8E2025=E5= =B9=B48=E6=9C=8813=E6=97=A5=E5=91=A8=E4=B8=89 00:14=E5=86=99=E9=81=93=EF=BC= =9A
On 8/12/25 01:24, px shi wrote:
>
>=C2=A0 =C2=A0 =C2=A01) What is the current archiving setup on the prima= ry and why is
>=C2=A0 =C2=A0 =C2=A0lagging?
>
>=C2=A0 =C2=A0The archive command uses pgBackRest to archive to S3. Beca= use it is
> uploaded to S3, the archiving speed is slow, which has caused lagging.=
>
>=C2=A0 =C2=A0 =C2=A02) Have you looked at archiving off the standby nod= e while it is in
>=C2=A0 =C2=A0 =C2=A0standby per:
>
> Yes, archiving on the standby node is disabled. Is it recommended to <= br> > share the WAL archive between the primary and standby nodes to avoid <= br> > interruptions in archiving?

Given that you are using a less then capable storage solution(S3) why do you think pushing the WAL from the standby to S3 would perform any
better then what is happening with the primary WAL?

The solution is to use a more capable storage platform.

>
> Adrian Klaver <adrian.klaver@aklaver.com
> <mailto:adrian.klaver@aklaver.com>> =E4=BA=8E2025=E5=B9=B48=E6=9C=888= =E6=97=A5=E5=91=A8=E4=BA=94 23:23=E5=86=99=E9=81=93=EF=BC=9A
>

--
Adrian Klaver
adrian.klave= r@aklaver.com
--0000000000004e7a8d063c359656--