Date: Sat, 17 Jul 2021 10:28:09 -0400
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: jeff.janes@gmail.com, mk@071.ovh, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #17103: WAL segments are not removed after exceeding
 max_slot_wal_keep_size
Message-ID: <202107171428.gawrkqbqbzzs@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="md26r72p6xn2lfxq"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <202107161809.zclasccpfcg3@alvherre.pgsql>
Archived-At: 
 <https://www.postgresql.org/message-id/202107171428.gawrkqbqbzzs%40alvherre.pgsql>
Precedence: bulk


--md26r72p6xn2lfxq
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On 2021-Jul-16, Alvaro Herrera wrote:

> The buildfarm has remained green so far, but clearly this is something
> we need to fix.  Maybe it's as simple as adding the loop we use below,
> starting at line 219.

There are a few failures of this on buildfarm now,

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=flaviventris&dt=2021-07-16%2022%3A30%3A35
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=desmoxytes&dt=2021-07-16%2021%3A52%3A04
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dragonet&dt=2021-07-16%2021%3A52%3A05

I am running the test in a loop with the attached; if it doesn't fail in
a few more rounds I'll push it.

There are two instances of a different failure:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=kittiwake&dt=2021-07-17%2013%3A39%3A43
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornet&dt=2021-07-16%2021%3A14%3A14

#   Failed test 'check that segments have been removed'
#   at t/019_replslot_limit.pl line 213.
#          got: '000000010000000000000021'
#     expected: '000000010000000000000022'
# Looks like you failed 1 test of 19.
[23:55:14] t/019_replslot_limit.pl .............. 
Dubious, test returned 1 (wstat 256, 0x100)

I'm afraid about this not being something we can fix with some
additional wait points ...

-- 
Álvaro Herrera              Valdivia, Chile  —  https://www.EnterpriseDB.com/
"La grandeza es una experiencia transitoria.  Nunca es consistente.
Depende en gran parte de la imaginación humana creadora de mitos"
(Irulan)

--md26r72p6xn2lfxq
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment; filename="loop.patch"

diff --git a/src/test/recovery/t/019_replslot_limit.pl b/src/test/recovery/t/019_replslot_limit.pl
index f1a00c67f2..cb94aca32a 100644
--- a/src/test/recovery/t/019_replslot_limit.pl
+++ b/src/test/recovery/t/019_replslot_limit.pl
@@ -177,18 +177,21 @@ ok( !find_in_log(
 my $logstart = get_log_size($node_primary);
 advance_wal($node_primary, 7);
 
-# This slot should be broken, wait for that to happen
-$node_primary->poll_query_until(
-	'postgres',
-	qq[SELECT wal_status = 'lost' FROM pg_replication_slots
-	WHERE slot_name = 'rep1']);
-
-# WARNING should be issued
-ok( find_in_log(
-		$node_primary,
-		"invalidating slot \"rep1\" because its restart_lsn [0-9A-F/]+ exceeds max_slot_wal_keep_size",
-		$logstart),
-	'check that the warning is logged');
+# wait until the WARNING is issued
+my $invalidated = 0;
+for (my $i = 0; $i < 10000; $i++)
+{
+	if (find_in_log(
+			$node_primary,
+			"invalidating slot \"rep1\" because its restart_lsn [0-9A-F/]+ exceeds max_slot_wal_keep_size",
+			$logstart))
+	{
+		$invalidated = 1;
+		last;
+	}
+	usleep(100_000);
+}
+ok($invalidated, 'check that the slot has been invalidated');
 
 $result = $node_primary->safe_psql(
 	'postgres',

--md26r72p6xn2lfxq--