MIME-Version: 1.0
From: Bala M <krishna.pgdba@gmail.com>
Date: Wed, 22 Oct 2025 14:33:06 +0530
Message-ID: <CAJ4rSwspyYBppQS701C=BVFX2yrP1JErgDCZXbC15BQh2zAS1Q@mail.gmail.com>
Subject: Index corruption issue after migration from RHEL 7 to RHEL 9
 (PostgreSQL 11 streaming replication)
To: pgsql-general@lists.postgresql.org
Content-Type: multipart/alternative; boundary="0000000000003d2a800641bb9623"
Archived-At: <https://www.postgresql.org/message-id/CAJ4rSwspyYBppQS701C%3DBVFX2yrP1JErgDCZXbC15BQh2zAS1Q%40mail.gmail.com>
Precedence: bulk

--0000000000003d2a800641bb9623
Content-Type: text/plain; charset="UTF-8"

Hi Team,

We are facing an issue related to index corruption after migrating our
PostgreSQL 11 setup from *RHEL 7* to *RHEL 9* using *streaming replication*
(base backup method).

After bringing up the standby on RHEL 9, we observed that certain tables
are not returning results when queries use indexed scans. Upon
investigation, the following check confirms index corruption:

The same indexes work fine on the RHEL 7 (primary) side. However, on
the RHEL 9 replica, queries that rely on this index return zero
records.
Rebuilding the indexes  fixed the issue temporarily but we have many
indexes  and our DB size is more than 10TB.

*Environment details:*

   -

   PostgreSQL Version: 11.15
   -

   OS on primary: RHEL 7.9
   -

   OS on standby: RHEL 9.6
   -

   Replication Type: Streaming replication (initialized using pg_basebackup)
   -

   Data Directory initialized from RHEL 7 base backup

*Issue Summary:*

   -

   Indexes appear and are the same size as per prod  on standby after base
   backup restore.
   -

   We are able to read  the data from the tables without index scans on
   standy by RHEL 9.
   -

   No filesystem or WAL errors observed in logs.

Could this be related to OS-level binary or page layout differences between
RHEL 7 and RHEL 9 for PostgreSQL 11 binaries?
Any insights or recommended actions would be greatly appreciated.


Thanks & Regards,
*krishna.*

--0000000000003d2a800641bb9623
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><p>Hi Team,</p>
<p>We are facing an issue related to index corruption after migrating our P=
ostgreSQL 11 setup from <strong>RHEL 7</strong> to <strong>RHEL 9</strong> =
using <strong>streaming replication</strong> (base backup method).</p>
<p>After bringing up the standby on RHEL 9, we observed that certain tables=
 are not returning results when queries use indexed scans. Upon investigati=
on, the following check confirms index corruption:</p>
<pre class=3D"gmail-overflow-visible!"><div class=3D"gmail-contain-inline-s=
ize gmail-rounded-2xl gmail-relative gmail-bg-token-sidebar-surface-primary=
"><div class=3D"gmail-sticky gmail-top-9"><div class=3D"gmail-absolute end-=
0 gmail-bottom-0 gmail-flex gmail-h-9 gmail-items-center gmail-pe-2"><div c=
lass=3D"gmail-bg-token-bg-elevated-secondary gmail-text-token-text-secondar=
y gmail-flex gmail-items-center gmail-gap-4 gmail-rounded-sm gmail-px-2 gma=
il-font-sans gmail-text-xs"></div></div></div><div class=3D"gmail-overflow-=
y-auto gmail-p-4" dir=3D"ltr"><span style=3D"font-family:Arial,Helvetica,sa=
ns-serif">The same indexes work fine on the RHEL 7 (primary) side. However,=
 on the RHEL 9 replica, queries that rely on this index return zero records=
. </span></div><div class=3D"gmail-overflow-y-auto gmail-p-4" dir=3D"ltr"><=
span style=3D"font-family:Arial,Helvetica,sans-serif">Rebuilding the indexe=
s </span><span style=3D"font-family:Arial,Helvetica,sans-serif"> fixed the =
issue temporarily but we have many indexes  and our DB size is more than 10=
TB.</span></div></div></pre>
<p><strong>Environment details:</strong></p>
<ul>
<li>
<p>PostgreSQL Version: 11.15</p>
</li>
<li>
<p>OS on primary: RHEL 7.9</p>
</li>
<li>
<p>OS on standby: RHEL 9.6</p>
</li>
<li>
<p>Replication Type: Streaming replication (initialized using <code>pg_base=
backup</code>)</p>
</li>
<li>
<p>Data Directory initialized from RHEL 7 base backup</p>
</li>
</ul>
<p><strong>Issue Summary:</strong></p>
<ul>
<li>
<p>Indexes appear and are the same=C2=A0size as per prod=C2=A0 on standby a=
fter base backup restore.</p>
</li>
<li>
<p>We are able to read=C2=A0 the data from the tables without index scans o=
n standy by RHEL 9.</p>
</li>
<li>
<p>No filesystem or WAL errors observed in logs.</p>
</li>
</ul>
<p>Could this be related to OS-level binary or page layout differences betw=
een RHEL 7 and RHEL 9 for PostgreSQL 11 binaries?<br>
Any insights or recommended actions would be greatly appreciated.</p><p><br=
></p>
<p>Thanks &amp; Regards,<br><b>krishna.</b></p></div><div><div dir=3D"ltr" =
class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"lt=
r"><div><br></div></div></div></div></div>

--0000000000003d2a800641bb9623--