public inbox for [email protected]  

help / color / mirror / Atom feed

From: Chris Wilson <[email protected]>
To: Bala M <[email protected]>
Cc: [email protected]
Subject: Re: Index corruption issue after migration from RHEL 7 to RHEL 9 (PostgreSQL 11 streaming replication)
Date: Wed, 22 Oct 2025 13:30:12 +0100
Message-ID: <CAOg7f80JJCMHXhNLUjnHMwBFV-f9Msv4L=PML0CdaZ4n47uEeA@mail.gmail.com> (raw)
In-Reply-To: <CAJ4rSwspyYBppQS701C=BVFX2yrP1JErgDCZXbC15BQh2zAS1Q@mail.gmail.com>
References: <CAJ4rSwspyYBppQS701C=BVFX2yrP1JErgDCZXbC15BQh2zAS1Q@mail.gmail.com>

Hi Krishna,

This might be due to the upgrade in glibc between RHEL7 and RHEL8, which
includes updated collations that change sort orders and thus invalidate
existing indexes, forcing them to be rebuilt (on a RHEL>7 system) if they
contain any values which sort differently under the new collation than the
old one.

More details can be found here, including a query to identify affected
indexes: https://wiki.postgresql.org/wiki/Locale_data_changes

[RHEL] *Version 8 uses the new locale data*. Therefore, caution will be
> necessary when upgrading."


Thanks, Chris.

On Wed, 22 Oct 2025 at 11:53, Bala M <[email protected]> wrote:

> Hi Team,
>
> We are facing an issue related to index corruption after migrating our
> PostgreSQL 11 setup from *RHEL 7* to *RHEL 9* using *streaming
> replication* (base backup method).
>
> After bringing up the standby on RHEL 9, we observed that certain tables
> are not returning results when queries use indexed scans. Upon
> investigation, the following check confirms index corruption:
>
> The same indexes work fine on the RHEL 7 (primary) side. However, on the RHEL 9 replica, queries that rely on this index return zero records.
> Rebuilding the indexes  fixed the issue temporarily but we have many indexes  and our DB size is more than 10TB.
>
> *Environment details:*
>
>    -
>
>    PostgreSQL Version: 11.15
>    -
>
>    OS on primary: RHEL 7.9
>    -
>
>    OS on standby: RHEL 9.6
>    -
>
>    Replication Type: Streaming replication (initialized using
>    pg_basebackup)
>    -
>
>    Data Directory initialized from RHEL 7 base backup
>
> *Issue Summary:*
>
>    -
>
>    Indexes appear and are the same size as per prod  on standby after
>    base backup restore.
>    -
>
>    We are able to read  the data from the tables without index scans on
>    standy by RHEL 9.
>    -
>
>    No filesystem or WAL errors observed in logs.
>
> Could this be related to OS-level binary or page layout differences
> between RHEL 7 and RHEL 9 for PostgreSQL 11 binaries?
> Any insights or recommended actions would be greatly appreciated.
>
>
> Thanks & Regards,
> *krishna.*
>
>

view thread (21+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Index corruption issue after migration from RHEL 7 to RHEL 9 (PostgreSQL 11 streaming replication)
  In-Reply-To: <CAOg7f80JJCMHXhNLUjnHMwBFV-f9Msv4L=PML0CdaZ4n47uEeA@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox