From: Heikki Linnakangas <hlinnaka@iki.fi>
To: g.shelkovy@arenadata.io
To: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18575: Sometimes pg_rewind mistakenly assumes that nothing needs to be done.
Date: Thu, 8 Aug 2024 10:03:29 +0300
Message-ID: <092dd515-b7b4-4fd0-8407-ceca2f02f6ec@iki.fi> (raw)
In-Reply-To: <18575-1b9b3b60e2a480de@postgresql.org>
References: <18575-1b9b3b60e2a480de@postgresql.org>
On 07/08/2024 15:19, PG Bug reporting form wrote:
> Sometimes pg_rewind mistakenly assumes that nothing needs to be done, which
> results in the replica having data that is not on the master.
>
> ...
> + pg_rewind --progress --debug '--source-pgdata=standby1'
> '--target-pgdata=standby2'
> pg_rewind: Source timeline history:
> pg_rewind: Target timeline history:
> pg_rewind: 1: 0/0 - 0/0
> pg_rewind: servers diverged at WAL location 0/10392E8 on timeline 1
> pg_rewind: no rewind required
>
> ...
> Same query
> + psql -d postgres -a --no-psqlrc -p 5001
> select count(*) from a;
> count
> --------
> 100000
> (1 row)
>
> + echo 'Different results'
> Different results
> + psql -d postgres -a --no-psqlrc -p 5002
> select count(*) from a;
> count
> --------
> 100340
> (1 row)
>
> + echo 'Stop standby1'
> Stop standby1
> + pg_ctl -w -D standby1 stop -m fast
> waiting for server to shut down....2024-08-07 17:15:03.077 +05 [19703]
> DEBUG: logger shutting down
> done
> server stopped
> + echo 'Stop standby2'
> Stop standby2
> + pg_ctl -w -D standby2 stop -m fast
> waiting for server to shut down....2024-08-07 17:15:03.117 +05 [19713]
> DEBUG: logger shutting down
> done
> server stopped
> ```
I cannot reproduce this. On my laptop, the reported "servers diverged at
WAL location" is always higher, and it performs rewind as expected:
g_rewind: Source timeline history:
pg_rewind: Target timeline history:
pg_rewind: 1: 0/0 - 0/0
pg_rewind: servers diverged at WAL location 0/1138F00 on timeline 1
pg_rewind: rewinding from last common checkpoint at 0/1138E88 on timeline 1
However while looking at the code, I noticed that the debug-output of
the Source timeline history is broken. See attached patch to fix it.
With that fix, I get:
pg_rewind: Source timeline history:
pg_rewind: 1: 0/0 - 0/1138F00
pg_rewind: 2: 0/1138F00 - 0/0
pg_rewind: Target timeline history:
pg_rewind: 1: 0/0 - 0/0
pg_rewind: 0 a: 1 0/0-0/1138F00 - b: 1 0/0-0/0
pg_rewind: servers diverged at WAL location 0/1138F00 on timeline 1
pg_rewind: rewinding from last common checkpoint at 0/1138E88 on timeline 1
That just fixes the debugging output, though, it doesn't change what it
actually does.
Can you try the attached fix show the output you get with that please?
Or alternatively, show the contents of the
standby1/pg_wal/00000002.history file.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
[text/x-patch] fix-print-debug-source-history.patch (519B, ../092dd515-b7b4-4fd0-8407-ceca2f02f6ec@iki.fi/2-fix-print-debug-source-history.patch)
download | inline diff:diff --git a/src/bin/pg_rewind/pg_rewind.c b/src/bin/pg_rewind/pg_rewind.cindex 0841ab4135..0f9d38abd5 100644--- a/src/bin/pg_rewind/pg_rewind.c+++ b/src/bin/pg_rewind/pg_rewind.c@@ -892,9 +892,9 @@ getTimelineHistory(TimeLineID tli, bool is_source, int *nentries)
pg_log_debug("Target timeline history:");
/*
- * Print the target timeline history.+ * Print the timeline history
*/
- for (i = 0; i < targetNentries; i++)+ for (i = 0; i < *nentries; i++)
{
TimeLineHistoryEntry *entry;
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: hlinnaka@iki.fi, g.shelkovy@arenadata.io, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18575: Sometimes pg_rewind mistakenly assumes that nothing needs to be done.
In-Reply-To: <092dd515-b7b4-4fd0-8407-ceca2f02f6ec@iki.fi>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox