MIME-Version: 1.0
References: <f9be7f2023784cbbb58255b0d9ea8281@ITUPW-EXMBOX2B.UniNet.unisa.edu.au>
 <CAB7nPqSHTf3Hb08+cdnFUprtNNLS0jLvzyivDMiV7+j0a2Lcng@mail.gmail.com>
In-Reply-To: <CAB7nPqSHTf3Hb08+cdnFUprtNNLS0jLvzyivDMiV7+j0a2Lcng@mail.gmail.com>
From: Bowen Shi <zxwsbg12138@gmail.com>
Date: Tue, 18 Feb 2025 14:47:09 +0800
Message-ID: <CAM_vCucirLRiB+cK3PWScfW=oCmBZz8oLYtwEufcMmHK-YzVsQ@mail.gmail.com>
Subject: Re: pg_rewind - enable wal_log_hints or data-checksums
To: Michael Paquier <michael.paquier@gmail.com>
Cc: Dylan Luong <Dylan.Luong@unisa.edu.au>, 
	"pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
Content-Type: multipart/alternative; boundary="0000000000002c8f32062e650322"
Archived-At: <https://www.postgresql.org/message-id/CAM_vCucirLRiB%2BcK3PWScfW%3DoCmBZz8oLYtwEufcMmHK-YzVsQ%40mail.gmail.com>
Precedence: bulk

--0000000000002c8f32062e650322
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Michael,

I first use initdb, and set wal_log_hints=3Doff, data_checksums=3Doff, and
full_page_writes=3Don. Starting pg and running for a while.

Then switch over happened, I used the following commands:
1. Old master postgresql.conf set wal_log_hints=3Don, then start and stop p=
g.
2. using  pg_rewind --target-pgdata=3DOldMaster --source-server=3DNewMaster=
 (no
error, old master's wal_log_hints has been set to on)

I wonder if this could lead to data corruption.


On Tue, Feb 18, 2025 at 2:27=E2=80=AFPM Michael Paquier <michael.paquier@gm=
ail.com>
wrote:

> On Mon, Jun 5, 2017 at 9:37 AM, Dylan Luong <Dylan.Luong@unisa.edu.au>
> wrote:
> > pg_rewind requires that the target server either has the wal_log_hints
> > option enabled in postgresql.conf or data checksums enabled when the
> cluster
> > was initialized with initdb.
>
> Yes, this is to make sure that you don't finish with a corrupted
> target server if a hint bit is set on a page after a checkpoint. Any
> of those options make sure that a full-page write is generated in this
> case.
>
> > What is the difference between the two options?
>
> Data checksums calculate 2 bytes of checksum data and write it to each
> page that is evicted from shared buffers. Each page read from disk has
> its checksum checked. In some workloads, like a heavy read load where
> a lot of page evictions happen, this can induce a couple of percents
> of performance lost. In my own experience, that's 1~2%.
>
> > What are the advantages and disadvantages between the two?
> > Which one is the the preferred option?
>
> If you care more about performance or if you use a file system that
> has its own block-level checksum, wal_log_hints would be preferred.
> Data checksums offer more guarantees in terms of integrity though when
> looking for corrupted data. Things get found more quickly.
> --
> Michael
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>
>
>

--0000000000002c8f32062e650322
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Michael,<br><br>I first use initdb, and set wal_log_hin=
ts=3Doff, data_checksums=3Doff, and full_page_writes=3Don. Starting pg and =
running for a while.<br><br>Then switch over happened, I used the following=
 commands:<br>1. Old master postgresql.conf set wal_log_hints=3Don, then st=
art and stop pg.=C2=A0<br>2. using =C2=A0pg_rewind --target-pgdata=3DOldMas=
ter --source-server=3DNewMaster (no error, old master&#39;s wal_log_hints h=
as been set to on)<br><div><br></div><div>I wonder if this could lead to da=
ta corruption.</div><div><br></div><div><br><div><br><div class=3D"gmail_qu=
ote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, Fe=
b 18, 2025 at 2:27=E2=80=AFPM Michael Paquier &lt;<a href=3D"mailto:michael=
.paquier@gmail.com">michael.paquier@gmail.com</a>&gt; wrote:<br></div><bloc=
kquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:=
1px solid rgb(204,204,204);padding-left:1ex">On Mon, Jun 5, 2017 at 9:37 AM=
, Dylan Luong &lt;<a href=3D"mailto:Dylan.Luong@unisa.edu.au" target=3D"_bl=
ank">Dylan.Luong@unisa.edu.au</a>&gt; wrote:<br>
&gt; pg_rewind requires that the target server either has the wal_log_hints=
<br>
&gt; option enabled in postgresql.conf or data checksums enabled when the c=
luster<br>
&gt; was initialized with initdb.<br>
<br>
Yes, this is to make sure that you don&#39;t finish with a corrupted<br>
target server if a hint bit is set on a page after a checkpoint. Any<br>
of those options make sure that a full-page write is generated in this<br>
case.<br>
<br>
&gt; What is the difference between the two options?<br>
<br>
Data checksums calculate 2 bytes of checksum data and write it to each<br>
page that is evicted from shared buffers. Each page read from disk has<br>
its checksum checked. In some workloads, like a heavy read load where<br>
a lot of page evictions happen, this can induce a couple of percents<br>
of performance lost. In my own experience, that&#39;s 1~2%.<br>
<br>
&gt; What are the advantages and disadvantages between the two?<br>
&gt; Which one is the the preferred option?<br>
<br>
If you care more about performance or if you use a file system that<br>
has its own block-level checksum, wal_log_hints would be preferred.<br>
Data checksums offer more guarantees in terms of integrity though when<br>
looking for corrupted data. Things get found more quickly.<br>
-- <br>
Michael<br>
<br>
<br>
-- <br>
Sent via pgsql-general mailing list (<a href=3D"mailto:pgsql-general@postgr=
esql.org" target=3D"_blank">pgsql-general@postgresql.org</a>)<br>
To make changes to your subscription:<br>
<a href=3D"http://www.postgresql.org/mailpref/pgsql-general" rel=3D"norefer=
rer" target=3D"_blank">http://www.postgresql.org/mailpref/pgsql-general</a>=
<br>
<br>
<br>
</blockquote></div></div></div></div>

--0000000000002c8f32062e650322--