public inbox for [email protected]
help / color / mirror / Atom feedRe: Test to dump and restore objects left behind by regression
4+ messages / 4 participants
[nested] [flat]
* Re: Test to dump and restore objects left behind by regression
@ 2025-04-03 08:20 Alvaro Herrera <[email protected]>
2025-04-03 09:14 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-04-03 13:40 ` Re: Test to dump and restore objects left behind by regression Andres Freund <[email protected]>
0 siblings, 2 replies; 4+ messages in thread
From: Alvaro Herrera @ 2025-04-03 08:20 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Apr-03, Ashutosh Bapat wrote:
> Looks like the problem is in the test itself as pointed out by Jeff in
> [1]. PFA patch fixing the test and enabling statistics back.
Thanks, pushed.
> A note about variable name changes and introduction of new variables.
> We run step 2 between 1 and 3 so that autovacuum gets a chance to run
> on the old cluster and update statistics. Autovacuum run is not
> necessary but useful here. Before these changes all the cluster
> initializations were using the same variables @initdb_params and
> %node_params. However with these changes, we modify the variable in
> step 2 and then again require original values in step 4. So I have
> used two sets of variables prefixed with old_ and new_ for clusters
> created in 1st step and 2nd step respectively. 4th step uses the
> variables with prefix old_. I think this change eliminates confusion
> caused by using same variables with different values.
This was a good change, thanks.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"No es bueno caminar con un hombre muerto"
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Test to dump and restore objects left behind by regression
2025-04-03 08:20 Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
@ 2025-04-03 09:14 ` Ashutosh Bapat <[email protected]>
1 sibling, 0 replies; 4+ messages in thread
From: Ashutosh Bapat @ 2025-04-03 09:14 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Apr 3, 2025 at 1:50 PM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Apr-03, Ashutosh Bapat wrote:
>
> > Looks like the problem is in the test itself as pointed out by Jeff in
> > [1]. PFA patch fixing the test and enabling statistics back.
>
> Thanks, pushed.
Thanks.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Test to dump and restore objects left behind by regression
2025-04-03 08:20 Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
@ 2025-04-03 13:40 ` Andres Freund <[email protected]>
1 sibling, 0 replies; 4+ messages in thread
From: Andres Freund @ 2025-04-03 13:40 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On 2025-04-03 10:20:09 +0200, Alvaro Herrera wrote:
> On 2025-Apr-03, Ashutosh Bapat wrote:
>
> > Looks like the problem is in the test itself as pointed out by Jeff in
> > [1]. PFA patch fixing the test and enabling statistics back.
>
> Thanks, pushed.
Since then the pg_upgrade tests have been failing on skink/valgrind, due to
exceeding the already substantially increased timeout.
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=skink&dt=2025-04-03%2007%3A06%3A19...
(note that there are other issues in that run)
284/333 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade TIMEOUT 10000.66s killed by signal 15 SIGTERM
[10:38:19.815](16.712s) ok 20 - check that locales in new cluster match original cluster
...
# Running: pg_dumpall --no-sync --dbname port=15114 host=/tmp/bh_AdT5uvQ dbname='postgres' --file /home/bf/bf-build/skink-master/HEAD/pgsql.build/testrun/pg_upgrade/002_pg_upgrade/data/tmp_test_gp2G/dump2.sql
death by signal at /home/bf/bf-build/skink-master/HEAD/pgsql/src/test/perl/PostgreSQL/Test/Cluster.pm line 181.
...
[10:44:11.720](351.905s) # Tests were run but no plan was declared and done_testing() was not seen.
I've increased the timeout even further, but I can't say that I am happy about
the slowest test getting even slower. Adding test time in the serially slowest
test is way worse than adding the same time in a concurrent test.
I suspect that the test will go a bit faster if log_statement weren't forced
on, printing that many log lines, with context, does make valgrind slower,
IME. But Cluster.pm forces it to on, and I suspect that putting a global
log_statement=false into TEMP_CONFIG would have it's own disadvantages.
/me and checks prices for increasing the size of skink's host.
Greetings,
Andres
^ permalink raw reply [nested|flat] 4+ messages in thread
* [PATCH v1 1/1] clean up AdjustUpgrade.pm after pg_upgrade min version bump
@ 2026-07-08 21:39 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Nathan Bossart @ 2026-07-08 21:39 UTC (permalink / raw)
---
.../perl/PostgreSQL/Test/AdjustUpgrade.pm | 207 +-----------------
1 file changed, 5 insertions(+), 202 deletions(-)
diff --git a/src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm b/src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
index 878fcc16a9e..04844298ab3 100644
--- a/src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
+++ b/src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
@@ -199,15 +199,10 @@ sub adjust_database_contents
'drop operator if exists #@%# (bigint,NONE)');
# get rid of dblink's dependencies on regress.so
- my $regrdb =
- $old_version le '9.4'
- ? 'contrib_regression'
- : 'contrib_regression_dblink';
-
- if ($dbnames{$regrdb})
+ if ($dbnames{'contrib_regression_dblink'})
{
_add_st(
- $result, $regrdb,
+ $result, 'contrib_regression_dblink',
'drop function if exists public.putenv(text)',
'drop function if exists public.wait_pid(integer)');
}
@@ -250,7 +245,7 @@ sub adjust_database_contents
}
# this table had OIDs too, but we'll just drop it
- if ($old_version >= 10 && $dbnames{'contrib_regression_postgres_fdw'})
+ if ($dbnames{'contrib_regression_postgres_fdw'})
{
_add_st(
$result,
@@ -276,43 +271,13 @@ sub adjust_database_contents
'drop function if exists public.funny_dup17()');
}
- # version-0 C functions are no longer supported
- if ($old_version < 10)
- {
- _add_st($result, 'regression',
- 'drop function oldstyle_length(integer, text)');
- }
-
- if ($old_version lt '9.5')
- {
- # cope with changes of underlying functions
- _add_st(
- $result,
- 'regression',
- 'drop operator @#@ (NONE, bigint)',
- 'CREATE OPERATOR @#@ ('
- . 'PROCEDURE = factorial, RIGHTARG = bigint )',
- 'drop aggregate public.array_cat_accum(anyarray)',
- 'CREATE AGGREGATE array_larger_accum (anyarray) ' . ' ( '
- . ' sfunc = array_larger, '
- . ' stype = anyarray, '
- . ' initcond = $${}$$ ' . ' ) ');
-
- # "=>" is no longer valid as an operator name
- _add_st($result, 'regression',
- 'drop operator if exists public.=> (bigint, NONE)');
- }
-
# Version 19 changed the output format of pg_lsn. To avoid output
# differences, set all pg_lsn columns to NULL if the old version is
# older than 19.
if ($old_version < 19)
{
- if ($old_version >= '9.5')
- {
- _add_st($result, 'regression',
- "update brintest set lsncol = NULL");
- }
+ _add_st($result, 'regression',
+ "update brintest set lsncol = NULL");
if ($old_version >= 12)
{
@@ -426,107 +391,6 @@ sub adjust_old_dumpfile
/$1 EXECUTE FUNCTION/mgx;
}
- # During pg_upgrade, we reindex hash indexes if the source is pre-v10.
- # This may change their tables' relallvisible values, so don't compare
- # those.
- if ($old_version < 10)
- {
- $dump =~ s/
- (^SELECT\s\*\sFROM\spg_catalog\.pg_restore_relation_stats\(
- [^;]*'relation',\s'public\.hash_[a-z0-9]*_heap'::regclass,
- [^;]*'relallvisible',)\s'\d+'::integer
- /$1 ''::integer/mgx;
- }
-
- if ($old_version lt '9.6')
- {
- # adjust some places where we don't print so many parens anymore
-
- my $prefix =
- "'New York'\tnew & york | big & apple | nyc\t'new' & 'york'\t";
- my $orig = "( 'new' & 'york' | 'big' & 'appl' ) | 'nyc'";
- my $repl = "'new' & 'york' | 'big' & 'appl' | 'nyc'";
- $dump =~ s/(?<=^\Q$prefix\E)\Q$orig\E/$repl/mg;
-
- $prefix =
- "'Sanct Peter'\tPeterburg | peter | 'Sanct Peterburg'\t'sanct' & 'peter'\t";
- $orig = "( 'peterburg' | 'peter' ) | 'sanct' & 'peterburg'";
- $repl = "'peterburg' | 'peter' | 'sanct' & 'peterburg'";
- $dump =~ s/(?<=^\Q$prefix\E)\Q$orig\E/$repl/mg;
- }
-
- if ($old_version lt '9.5')
- {
- # adjust some places where we don't print so many parens anymore
-
- my $prefix = "CONSTRAINT (?:sequence|copy)_con CHECK [(][(]";
- my $orig = "((x > 3) AND (y <> 'check failed'::text))";
- my $repl = "(x > 3) AND (y <> 'check failed'::text)";
- $dump =~ s/($prefix)\Q$orig\E/$1$repl/mg;
-
- $prefix = "CONSTRAINT insert_con CHECK [(][(]";
- $orig = "((x >= 3) AND (y <> 'check failed'::text))";
- $repl = "(x >= 3) AND (y <> 'check failed'::text)";
- $dump =~ s/($prefix)\Q$orig\E/$1$repl/mg;
-
- $orig = "DEFAULT ((-1) * currval('public.insert_seq'::regclass))";
- $repl =
- "DEFAULT ('-1'::integer * currval('public.insert_seq'::regclass))";
- $dump =~ s/\Q$orig\E/$repl/mg;
-
- my $expr =
- "(rsl.sl_color = rsh.slcolor) AND (rsl.sl_len_cm >= rsh.slminlen_cm)";
- $dump =~ s/WHERE \(\(\Q$expr\E\)/WHERE ($expr/g;
-
- $expr =
- "(rule_and_refint_t3.id3a = new.id3a) AND (rule_and_refint_t3.id3b = new.id3b)";
- $dump =~ s/WHERE \(\(\Q$expr\E\)/WHERE ($expr/g;
-
- $expr =
- "(rule_and_refint_t3_1.id3a = new.id3a) AND (rule_and_refint_t3_1.id3b = new.id3b)";
- $dump =~ s/WHERE \(\(\Q$expr\E\)/WHERE ($expr/g;
- }
-
- if ($old_version lt '9.3')
- {
- # CREATE VIEW/RULE statements were not pretty-printed before 9.3.
- # To cope, reduce all whitespace sequences within them to one space.
- # This must be done on both old and new dumps.
- $dump = _mash_view_whitespace($dump);
-
- # _mash_view_whitespace doesn't handle multi-command rules;
- # rather than trying to fix that, just hack the exceptions manually.
-
- my $prefix =
- "CREATE RULE rtest_sys_del AS ON DELETE TO public.rtest_system DO (DELETE FROM public.rtest_interface WHERE (rtest_interface.sysname = old.sysname);";
- my $line2 = " DELETE FROM public.rtest_admin";
- my $line3 = " WHERE (rtest_admin.sysname = old.sysname);";
- $dump =~
- s/(?<=\Q$prefix\E)\Q$line2$line3\E \);/\n$line2\n $line3\n);/mg;
-
- $prefix =
- "CREATE RULE rtest_sys_upd AS ON UPDATE TO public.rtest_system DO (UPDATE public.rtest_interface SET sysname = new.sysname WHERE (rtest_interface.sysname = old.sysname);";
- $line2 = " UPDATE public.rtest_admin SET sysname = new.sysname";
- $line3 = " WHERE (rtest_admin.sysname = old.sysname);";
- $dump =~
- s/(?<=\Q$prefix\E)\Q$line2$line3\E \);/\n$line2\n $line3\n);/mg;
-
- # and there's one place where pre-9.3 uses a different table alias
- $dump =~ s {^(CREATE\sRULE\srule_and_refint_t3_ins\sAS\s
- ON\sINSERT\sTO\spublic\.rule_and_refint_t3\s
- WHERE\s\(EXISTS\s\(SELECT\s1\sFROM\spublic\.rule_and_refint_t3)\s
- (WHERE\s\(\(rule_and_refint_t3)
- (\.id3a\s=\snew\.id3a\)\sAND\s\(rule_and_refint_t3)
- (\.id3b\s=\snew\.id3b\)\sAND\s\(rule_and_refint_t3)}
- {$1 rule_and_refint_t3_1 $2_1$3_1$4_1}mx;
-
- # Also fix old use of NATURAL JOIN syntax
- $dump =~ s {NATURAL JOIN public\.credit_card r}
- {JOIN public.credit_card r USING (cid)}mg;
- $dump =~ s {NATURAL JOIN public\.credit_usage r}
- {JOIN public.credit_usage r USING (cid)}mg;
- }
-
# Suppress blank lines, as some places in pg_dump emit more or fewer.
$dump =~ s/\n\n+/\n/g;
@@ -661,37 +525,6 @@ sub _mash_view_qualifiers
return $dump;
}
-
-# Internal subroutine to mangle whitespace within view/rule commands.
-# Any consecutive sequence of whitespace is reduced to one space.
-sub _mash_view_whitespace
-{
- my ($dump) = @_;
-
- foreach my $leader ('CREATE VIEW', 'CREATE RULE')
- {
- my @splitchunks = split $leader, $dump;
-
- $dump = shift(@splitchunks);
- foreach my $chunk (@splitchunks)
- {
- my @thischunks = split /;/, $chunk, 2;
- my $stmt = shift(@thischunks);
-
- # now $stmt is just the body of the CREATE VIEW/RULE
- $stmt =~ s/\s+/ /sg;
- # we also need to smash these forms for sub-selects and rules
- $stmt =~ s/\( SELECT/(SELECT/g;
- $stmt =~ s/\( INSERT/(INSERT/g;
- $stmt =~ s/\( UPDATE/(UPDATE/g;
- $stmt =~ s/\( DELETE/(DELETE/g;
-
- $dump .= $leader . $stmt . ';' . $thischunks[0];
- }
- }
- return $dump;
-}
-
=pod
=item adjust_new_dumpfile($old_version, $dump)
@@ -779,36 +612,6 @@ sub adjust_new_dumpfile
$dump =~ s/^SET default_table_access_method = heap;\n//mg;
}
- # During pg_upgrade, we reindex hash indexes if the source is pre-v10.
- # This may change their tables' relallvisible values, so don't compare
- # those.
- if ($old_version < 10)
- {
- $dump =~ s/
- (^SELECT\s\*\sFROM\spg_catalog\.pg_restore_relation_stats\(
- [^;]*'relation',\s'public\.hash_[a-z0-9]*_heap'::regclass,
- [^;]*'relallvisible',)\s'\d+'::integer
- /$1 ''::integer/mgx;
- }
-
- # dumps from pre-9.6 dblink may include redundant ACL settings
- if ($old_version lt '9.6')
- {
- my $comment =
- "-- Name: FUNCTION dblink_connect_u\(.*?\); Type: ACL; Schema: public; Owner: .*";
- my $sql =
- "REVOKE ALL ON FUNCTION public\.dblink_connect_u\(.*?\) FROM PUBLIC;";
- $dump =~ s/^--\n$comment\n--\n+$sql\n+//mg;
- }
-
- if ($old_version lt '9.3')
- {
- # CREATE VIEW/RULE statements were not pretty-printed before 9.3.
- # To cope, reduce all whitespace sequences within them to one space.
- # This must be done on both old and new dumps.
- $dump = _mash_view_whitespace($dump);
- }
-
# Suppress blank lines, as some places in pg_dump emit more or fewer.
$dump =~ s/\n\n+/\n/g;
--
2.50.1 (Apple Git-155)
--WhtY6+QncUkBAPlR--
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-07-08 21:39 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-04-03 08:20 Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-04-03 09:14 ` Ashutosh Bapat <[email protected]>
2025-04-03 13:40 ` Andres Freund <[email protected]>
2026-07-08 21:39 [PATCH v1 1/1] clean up AdjustUpgrade.pm after pg_upgrade min version bump Nathan Bossart <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox