Message-ID: <7161562f-d00d-41a6-bd8a-d15684d3eaf2@gmail.com>
Date: Sun, 27 Apr 2025 17:00:01 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: optimize file transfer in pg_upgrade
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>,
 Andres Freund <andres@anarazel.de>, Greg Sabino Mullane
 <htamfids@gmail.com>, pgsql-hackers@postgresql.org, bruce@momjian.us
References: <tlkgkf3woe4ye2l4m4dnwui4wmqtajotozw5r3fhefyuv6hwrl@34xroqbvddpj>
 <Z9mxlf5uVuk2DziP@nathan>
 <7rhxl5nl5cuenm4kzprjhaojo74m2teuwzjffcrdi6bxx6b2wn@orb5h3mteh2y>
 <Z9nEup6k56N_9l0X@nathan> <Z9oofphRMQQzgRmL@nathan> <Z9rjSmA_MpPz-zOw@nathan>
 <712605.1742401713@sss.pgh.pa.us>
 <can7m7lntp3rpwchr7upnlom4fhzwfwttfa53xks2yxxafxfkv@3nzrihnqmx4g>
 <Z9sbsU4cu7O9glEQ@nathan> <Z9s29-mO6wnSEU_r@nathan> <Z9t3QsiVz2jHw76A@nathan>
Content-Language: en-US
From: Alexander Lakhin <exclusion@gmail.com>
In-Reply-To: <Z9t3QsiVz2jHw76A@nathan>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/7161562f-d00d-41a6-bd8a-d15684d3eaf2%40gmail.com>
Precedence: bulk

Hello Nathan,

20.03.2025 04:02, Nathan Bossart wrote:
> On Wed, Mar 19, 2025 at 04:28:23PM -0500, Nathan Bossart wrote:
> And here is yet another new version of the full patch set.  I'm planning to
> commit 0001 (the new pg_upgrade transfer mode test) tomorrow so that I can
> deal with any buildfarm indigestion before committing swap mode.  I did run
> the test locally for upgrades from v9.6, v13, and v17, but who knows what
> unique configurations I've failed to anticipate...

I found a couple of the 006_transfer_modes failures during the past month:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2025-04-08%2004%3A18%3A15
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2025-04-21%2008%3A03%3A06

Both happened on Windows, but what's worse is that the failure logs
contain no information on the exact reason. We can see:
#   Failed test 'pg_upgrade with transfer mode --swap: stdout matches'
#   at C:/tools/xmsys64/home/pgrunner/bf/root/HEAD/pgsql/src/bin/pg_upgrade/t/006_transfer_modes.pl line 61.
...
# Restoring database schemas in the new cluster
# *failure*
#
# Consult the last few lines of 
"C:/tools/xmsys64/home/pgrunner/bf/root/HEAD/pgsql.build/testrun/pg_upgrade/006_transfer_modes/data/t_006_transfer_modes_new_data/pgdata/pg_upgrade_output.d/20250421T081115.575/log/pg_upgrade_dump_1.log" 
for
# the probable cause of the failure.
# Failure, exiting
# '
#     doesn't match '(?^:.* not supported on this platform|could not .* between old and new data directories: .*)'

there is a reference to pg_upgrade_dump_x.log, but no such files saved.

I tried to reproduce this failure locally, but failed. Still I discovered
that when the test fails, the target directory containing pgdata/ gets
removed, because of this coding:
     my $result = command_ok_or_fails_like(
...
     # If pg_upgrade was successful, check that all of our test objects reached
     # the new version.
     if ($result)
     {
...
     }

     $old->clean_node();
     $new->clean_node();

Moreover, even when pg_upgrade succeeds, IPC::Run::run inside
command_ok_or_fails_like() returns false, as we can see from a
successful test run:
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=fairywren&dt=2025-04-27%2001%3A03%3A06&stg=misc-check

pgsql.build/testrun/pg_upgrade/006_transfer_modes/log/regress_log_006_transfer_modes
[01:18:38.210](21.036s) ok 1 - pg_upgrade with transfer mode --clone: stdout matches
[01:18:38.211](0.001s) ok 2 - pg_upgrade with transfer mode --clone: stderr matches

The corresponding code is:
     print("# Running: " . join(" ", @{$cmd}) . "\n");
     my $result = IPC::Run::run $cmd, '>' => \$stdout, '2>' => \$stderr;
     if (!$result)
     {
         like($stdout, $expected_stdout, "$test_name: stdout matches");
         like($stderr, $expected_stderr, "$test_name: stderr matches");
     }

So maybe it's worth to adjust the test somehow to have interesting logs
left after a failure?

Best regards,
Alexander Lakhin
Neon (https://neon.tech)