Feedback-ID: i0fe9450f:Fastmail
Date: Mon, 6 Jun 2022 13:58:40 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Andres Freund <andres@anarazel.de>
Cc: Noah Misch <noah@leadboat.com>, Tom Lane <tgl@sss.pgh.pa.us>,
	Thomas Munro <thomas.munro@gmail.com>,
	Julien Rouhaud <rjuju123@gmail.com>,
	Andrew Dunstan <andrew@dunslane.net>,
	Postgres hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Rewriting the test of pg_upgrade as a TAP test - take three -
 remastered set
Message-ID: <Yp2JgI7DDXkWBjYN@paquier.xyz>
References: <20220511053255.GA2578591@rfd.leadboat.com>
 <YnyawnlheodeuFfS@paquier.xyz>
 <20220514082728.GE2684227@rfd.leadboat.com>
 <YoHhWD5vQzb2mmiF@paquier.xyz>
 <20220518080315.GC2820845@rfd.leadboat.com>
 <YoS6SFWXvI5zplXV@paquier.xyz>
 <20220521012801.GA2934331@rfd.leadboat.com>
 <YohWm/hZHxNMpvyV@paquier.xyz>
 <20220603195318.qk4voicqfdhlsnoh@alap3.anarazel.de>
 <YprTEev9lz5NBbxZ@paquier.xyz>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="Wv8YzMPk3+sIZDMq"
Content-Disposition: inline
In-Reply-To: <YprTEev9lz5NBbxZ@paquier.xyz>
Archived-At: 
 <https://www.postgresql.org/message-id/Yp2JgI7DDXkWBjYN%40paquier.xyz>
Precedence: bulk


--Wv8YzMPk3+sIZDMq
Content-Type: multipart/mixed; boundary="iZoEUx1IalvQPeGA"
Content-Disposition: inline


--iZoEUx1IalvQPeGA
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Sat, Jun 04, 2022 at 12:35:45PM +0900, Michael Paquier wrote:
> Something like 80~85% of the bloat comes from the diffs in your case.
> Well, it is always possible to limit that to an arbitrary amount of
> characters (say 50k~100k?) to still give some context, and dump the
> whole in a different file outside the log/ path (aka tmp_check/), so
> that the buildfarm would show a minimum amount of information, while
> local failures would still have an access to everything.

After looking a bit around that.  Something like the attached, where
the characters are limited at 10k, would limit the output generated..
--
Michael

--iZoEUx1IalvQPeGA
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="upgrade-tap-logs.patch"
Content-Transfer-Encoding: quoted-printable

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/=
002_pg_upgrade.pl
index 55c7354ba2..4581ddc915 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -223,6 +223,9 @@ command_ok(
 	'run of pg_upgrade for new instance');
 $newnode->start;
=20
+# Limit all contents to 10k characters.
+my $report_max_chars =3D 10000;
+
 # Check if there are any logs coming from pg_upgrade, that would only be
 # retained on failure.
 my $log_path =3D $newnode->data_dir . "/pg_upgrade_output.d/log";
@@ -231,7 +234,8 @@ if (-d $log_path)
 	foreach my $log (glob("$log_path/*"))
 	{
 		note "=3D=3D=3D contents of $log =3D=3D=3D\n";
-		print slurp_file($log);
+		my $contents =3D slurp_file($log);
+		print substr($contents, 0, $report_max_chars);
 		print "=3D=3D=3D EOF =3D=3D=3D\n";
 	}
 }
@@ -256,9 +260,9 @@ if ($compare_res !=3D 0)
 	  run_command([ 'diff', "$tempdir/dump1.sql", "$tempdir/dump2.sql" ]);
 	print "=3D=3D=3D diff of $tempdir/dump1.sql and $tempdir/dump2.sql\n";
 	print "=3D=3D=3D stdout =3D=3D=3D\n";
-	print $stdout;
+	print substr($stdout, 0, $report_max_chars);
 	print "=3D=3D=3D stderr =3D=3D=3D\n";
-	print $stderr;
+	print substr($stderr, 0, $report_max_chars);
 	print "=3D=3D=3D EOF =3D=3D=3D\n";
 }
=20

--iZoEUx1IalvQPeGA--

--Wv8YzMPk3+sIZDMq
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmKdiYAACgkQnvQgOdby
QH1zig//Z4qbt1RC26gujSOUGBCE1o0ywA1HIEVkpf/ytXEOphtuqyogkBAFlgnn
EKNRDCy7e8yfhe3GdBdIYBXfaR5QqOAomuY5/orvcRd/fsRyWhhRLf/fS87csULe
1hOwjMv5xdiyXVVhfJkQlbCkG7oUIkgzcHAT18vW7zdUYYqqvdmrRhQxuTlPbUBv
AqysCeJGsmGdwvrRbWvE66znfTeo8ZD34jl5cJKKD+X8iHMI1k22npm32omLgqFY
3wo6Q5qJJAEhwxAePZK2x8fyXPiC2eRJkKSa7TKj+Dv80rK+qbt8SojBam2gx8zX
ywabs9aYO92D16hxH/jiUNCW3FoCvOhSS17YQcS4EpOEFgr2ZnVfhgB/dN1DSCOS
IyXUdIQhSNEYD/tcZoScnRCSsf+MQdhJzNh1488RaAgzon6HfDnCMn6t/1ek0MvX
h3N/eA3o2VUKFTCA9VhG4oMnskylmWrU8AtE7m/Af0N7v4RZQR6T05WcBSjyKHkJ
Lgkhq/0RbU8cPrYbZNMhADcZkcWjrjmCnaaz1zy71e48LHNG5dIVg8wFOXx6l5b7
EPjm69L3klWa3yEeg/gY76+BtxfikrEuEm6mbZez+oofSmPMEClF6RQv7UC9PQt/
uusK3Gef6+jjm0nMsy9Zce/uokiJoergtLc/4BkR4K9x3pqp9HI=
=0Kot
-----END PGP SIGNATURE-----

--Wv8YzMPk3+sIZDMq--