public inbox for [email protected]help / color / mirror / Atom feed
Re: Test to dump and restore objects left behind by regression 4+ messages / 4 participants [nested] [flat]
* Re: Test to dump and restore objects left behind by regression @ 2025-04-03 08:20 Alvaro Herrera <[email protected]> 0 siblings, 2 replies; 4+ messages in thread From: Alvaro Herrera @ 2025-04-03 08:20 UTC (permalink / raw) To: Ashutosh Bapat <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]> On 2025-Apr-03, Ashutosh Bapat wrote: > Looks like the problem is in the test itself as pointed out by Jeff in > [1]. PFA patch fixing the test and enabling statistics back. Thanks, pushed. > A note about variable name changes and introduction of new variables. > We run step 2 between 1 and 3 so that autovacuum gets a chance to run > on the old cluster and update statistics. Autovacuum run is not > necessary but useful here. Before these changes all the cluster > initializations were using the same variables @initdb_params and > %node_params. However with these changes, we modify the variable in > step 2 and then again require original values in step 4. So I have > used two sets of variables prefixed with old_ and new_ for clusters > created in 1st step and 2nd step respectively. 4th step uses the > variables with prefix old_. I think this change eliminates confusion > caused by using same variables with different values. This was a good change, thanks. -- Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/ "No es bueno caminar con un hombre muerto" ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Test to dump and restore objects left behind by regression @ 2025-04-03 09:14 Ashutosh Bapat <[email protected]> parent: Alvaro Herrera <[email protected]> 1 sibling, 0 replies; 4+ messages in thread From: Ashutosh Bapat @ 2025-04-03 09:14 UTC (permalink / raw) To: Alvaro Herrera <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]> On Thu, Apr 3, 2025 at 1:50 PM Alvaro Herrera <[email protected]> wrote: > > On 2025-Apr-03, Ashutosh Bapat wrote: > > > Looks like the problem is in the test itself as pointed out by Jeff in > > [1]. PFA patch fixing the test and enabling statistics back. > > Thanks, pushed. Thanks. -- Best Wishes, Ashutosh Bapat ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Test to dump and restore objects left behind by regression @ 2025-04-03 13:40 Andres Freund <[email protected]> parent: Alvaro Herrera <[email protected]> 1 sibling, 0 replies; 4+ messages in thread From: Andres Freund @ 2025-04-03 13:40 UTC (permalink / raw) To: Alvaro Herrera <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]> Hi, On 2025-04-03 10:20:09 +0200, Alvaro Herrera wrote: > On 2025-Apr-03, Ashutosh Bapat wrote: > > > Looks like the problem is in the test itself as pointed out by Jeff in > > [1]. PFA patch fixing the test and enabling statistics back. > > Thanks, pushed. Since then the pg_upgrade tests have been failing on skink/valgrind, due to exceeding the already substantially increased timeout. https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=skink&dt=2025-04-03%2007%3A06%3A19... (note that there are other issues in that run) 284/333 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade TIMEOUT 10000.66s killed by signal 15 SIGTERM [10:38:19.815](16.712s) ok 20 - check that locales in new cluster match original cluster ... # Running: pg_dumpall --no-sync --dbname port=15114 host=/tmp/bh_AdT5uvQ dbname='postgres' --file /home/bf/bf-build/skink-master/HEAD/pgsql.build/testrun/pg_upgrade/002_pg_upgrade/data/tmp_test_gp2G/dump2.sql death by signal at /home/bf/bf-build/skink-master/HEAD/pgsql/src/test/perl/PostgreSQL/Test/Cluster.pm line 181. ... [10:44:11.720](351.905s) # Tests were run but no plan was declared and done_testing() was not seen. I've increased the timeout even further, but I can't say that I am happy about the slowest test getting even slower. Adding test time in the serially slowest test is way worse than adding the same time in a concurrent test. I suspect that the test will go a bit faster if log_statement weren't forced on, printing that many log lines, with context, does make valgrind slower, IME. But Cluster.pm forces it to on, and I suspect that putting a global log_statement=false into TEMP_CONFIG would have it's own disadvantages. /me and checks prices for increasing the size of skink's host. Greetings, Andres ^ permalink raw reply [nested|flat] 4+ messages in thread
* [PATCH v1 1/1] clean up AdjustUpgrade.pm after pg_upgrade min version bump @ 2026-07-08 21:39 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Nathan Bossart @ 2026-07-08 21:39 UTC (permalink / raw) --- .../perl/PostgreSQL/Test/AdjustUpgrade.pm | 207 +----------------- 1 file changed, 5 insertions(+), 202 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm b/src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm index 878fcc16a9e..04844298ab3 100644 --- a/src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm +++ b/src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm @@ -199,15 +199,10 @@ sub adjust_database_contents 'drop operator if exists #@%# (bigint,NONE)'); # get rid of dblink's dependencies on regress.so - my $regrdb = - $old_version le '9.4' - ? 'contrib_regression' - : 'contrib_regression_dblink'; - - if ($dbnames{$regrdb}) + if ($dbnames{'contrib_regression_dblink'}) { _add_st( - $result, $regrdb, + $result, 'contrib_regression_dblink', 'drop function if exists public.putenv(text)', 'drop function if exists public.wait_pid(integer)'); } @@ -250,7 +245,7 @@ sub adjust_database_contents } # this table had OIDs too, but we'll just drop it - if ($old_version >= 10 && $dbnames{'contrib_regression_postgres_fdw'}) + if ($dbnames{'contrib_regression_postgres_fdw'}) { _add_st( $result, @@ -276,43 +271,13 @@ sub adjust_database_contents 'drop function if exists public.funny_dup17()'); } - # version-0 C functions are no longer supported - if ($old_version < 10) - { - _add_st($result, 'regression', - 'drop function oldstyle_length(integer, text)'); - } - - if ($old_version lt '9.5') - { - # cope with changes of underlying functions - _add_st( - $result, - 'regression', - 'drop operator @#@ (NONE, bigint)', - 'CREATE OPERATOR @#@ (' - . 'PROCEDURE = factorial, RIGHTARG = bigint )', - 'drop aggregate public.array_cat_accum(anyarray)', - 'CREATE AGGREGATE array_larger_accum (anyarray) ' . ' ( ' - . ' sfunc = array_larger, ' - . ' stype = anyarray, ' - . ' initcond = $${}$$ ' . ' ) '); - - # "=>" is no longer valid as an operator name - _add_st($result, 'regression', - 'drop operator if exists public.=> (bigint, NONE)'); - } - # Version 19 changed the output format of pg_lsn. To avoid output # differences, set all pg_lsn columns to NULL if the old version is # older than 19. if ($old_version < 19) { - if ($old_version >= '9.5') - { - _add_st($result, 'regression', - "update brintest set lsncol = NULL"); - } + _add_st($result, 'regression', + "update brintest set lsncol = NULL"); if ($old_version >= 12) { @@ -426,107 +391,6 @@ sub adjust_old_dumpfile /$1 EXECUTE FUNCTION/mgx; } - # During pg_upgrade, we reindex hash indexes if the source is pre-v10. - # This may change their tables' relallvisible values, so don't compare - # those. - if ($old_version < 10) - { - $dump =~ s/ - (^SELECT\s\*\sFROM\spg_catalog\.pg_restore_relation_stats\( - [^;]*'relation',\s'public\.hash_[a-z0-9]*_heap'::regclass, - [^;]*'relallvisible',)\s'\d+'::integer - /$1 ''::integer/mgx; - } - - if ($old_version lt '9.6') - { - # adjust some places where we don't print so many parens anymore - - my $prefix = - "'New York'\tnew & york | big & apple | nyc\t'new' & 'york'\t"; - my $orig = "( 'new' & 'york' | 'big' & 'appl' ) | 'nyc'"; - my $repl = "'new' & 'york' | 'big' & 'appl' | 'nyc'"; - $dump =~ s/(?<=^\Q$prefix\E)\Q$orig\E/$repl/mg; - - $prefix = - "'Sanct Peter'\tPeterburg | peter | 'Sanct Peterburg'\t'sanct' & 'peter'\t"; - $orig = "( 'peterburg' | 'peter' ) | 'sanct' & 'peterburg'"; - $repl = "'peterburg' | 'peter' | 'sanct' & 'peterburg'"; - $dump =~ s/(?<=^\Q$prefix\E)\Q$orig\E/$repl/mg; - } - - if ($old_version lt '9.5') - { - # adjust some places where we don't print so many parens anymore - - my $prefix = "CONSTRAINT (?:sequence|copy)_con CHECK [(][(]"; - my $orig = "((x > 3) AND (y <> 'check failed'::text))"; - my $repl = "(x > 3) AND (y <> 'check failed'::text)"; - $dump =~ s/($prefix)\Q$orig\E/$1$repl/mg; - - $prefix = "CONSTRAINT insert_con CHECK [(][(]"; - $orig = "((x >= 3) AND (y <> 'check failed'::text))"; - $repl = "(x >= 3) AND (y <> 'check failed'::text)"; - $dump =~ s/($prefix)\Q$orig\E/$1$repl/mg; - - $orig = "DEFAULT ((-1) * currval('public.insert_seq'::regclass))"; - $repl = - "DEFAULT ('-1'::integer * currval('public.insert_seq'::regclass))"; - $dump =~ s/\Q$orig\E/$repl/mg; - - my $expr = - "(rsl.sl_color = rsh.slcolor) AND (rsl.sl_len_cm >= rsh.slminlen_cm)"; - $dump =~ s/WHERE \(\(\Q$expr\E\)/WHERE ($expr/g; - - $expr = - "(rule_and_refint_t3.id3a = new.id3a) AND (rule_and_refint_t3.id3b = new.id3b)"; - $dump =~ s/WHERE \(\(\Q$expr\E\)/WHERE ($expr/g; - - $expr = - "(rule_and_refint_t3_1.id3a = new.id3a) AND (rule_and_refint_t3_1.id3b = new.id3b)"; - $dump =~ s/WHERE \(\(\Q$expr\E\)/WHERE ($expr/g; - } - - if ($old_version lt '9.3') - { - # CREATE VIEW/RULE statements were not pretty-printed before 9.3. - # To cope, reduce all whitespace sequences within them to one space. - # This must be done on both old and new dumps. - $dump = _mash_view_whitespace($dump); - - # _mash_view_whitespace doesn't handle multi-command rules; - # rather than trying to fix that, just hack the exceptions manually. - - my $prefix = - "CREATE RULE rtest_sys_del AS ON DELETE TO public.rtest_system DO (DELETE FROM public.rtest_interface WHERE (rtest_interface.sysname = old.sysname);"; - my $line2 = " DELETE FROM public.rtest_admin"; - my $line3 = " WHERE (rtest_admin.sysname = old.sysname);"; - $dump =~ - s/(?<=\Q$prefix\E)\Q$line2$line3\E \);/\n$line2\n $line3\n);/mg; - - $prefix = - "CREATE RULE rtest_sys_upd AS ON UPDATE TO public.rtest_system DO (UPDATE public.rtest_interface SET sysname = new.sysname WHERE (rtest_interface.sysname = old.sysname);"; - $line2 = " UPDATE public.rtest_admin SET sysname = new.sysname"; - $line3 = " WHERE (rtest_admin.sysname = old.sysname);"; - $dump =~ - s/(?<=\Q$prefix\E)\Q$line2$line3\E \);/\n$line2\n $line3\n);/mg; - - # and there's one place where pre-9.3 uses a different table alias - $dump =~ s {^(CREATE\sRULE\srule_and_refint_t3_ins\sAS\s - ON\sINSERT\sTO\spublic\.rule_and_refint_t3\s - WHERE\s\(EXISTS\s\(SELECT\s1\sFROM\spublic\.rule_and_refint_t3)\s - (WHERE\s\(\(rule_and_refint_t3) - (\.id3a\s=\snew\.id3a\)\sAND\s\(rule_and_refint_t3) - (\.id3b\s=\snew\.id3b\)\sAND\s\(rule_and_refint_t3)} - {$1 rule_and_refint_t3_1 $2_1$3_1$4_1}mx; - - # Also fix old use of NATURAL JOIN syntax - $dump =~ s {NATURAL JOIN public\.credit_card r} - {JOIN public.credit_card r USING (cid)}mg; - $dump =~ s {NATURAL JOIN public\.credit_usage r} - {JOIN public.credit_usage r USING (cid)}mg; - } - # Suppress blank lines, as some places in pg_dump emit more or fewer. $dump =~ s/\n\n+/\n/g; @@ -661,37 +525,6 @@ sub _mash_view_qualifiers return $dump; } - -# Internal subroutine to mangle whitespace within view/rule commands. -# Any consecutive sequence of whitespace is reduced to one space. -sub _mash_view_whitespace -{ - my ($dump) = @_; - - foreach my $leader ('CREATE VIEW', 'CREATE RULE') - { - my @splitchunks = split $leader, $dump; - - $dump = shift(@splitchunks); - foreach my $chunk (@splitchunks) - { - my @thischunks = split /;/, $chunk, 2; - my $stmt = shift(@thischunks); - - # now $stmt is just the body of the CREATE VIEW/RULE - $stmt =~ s/\s+/ /sg; - # we also need to smash these forms for sub-selects and rules - $stmt =~ s/\( SELECT/(SELECT/g; - $stmt =~ s/\( INSERT/(INSERT/g; - $stmt =~ s/\( UPDATE/(UPDATE/g; - $stmt =~ s/\( DELETE/(DELETE/g; - - $dump .= $leader . $stmt . ';' . $thischunks[0]; - } - } - return $dump; -} - =pod =item adjust_new_dumpfile($old_version, $dump) @@ -779,36 +612,6 @@ sub adjust_new_dumpfile $dump =~ s/^SET default_table_access_method = heap;\n//mg; } - # During pg_upgrade, we reindex hash indexes if the source is pre-v10. - # This may change their tables' relallvisible values, so don't compare - # those. - if ($old_version < 10) - { - $dump =~ s/ - (^SELECT\s\*\sFROM\spg_catalog\.pg_restore_relation_stats\( - [^;]*'relation',\s'public\.hash_[a-z0-9]*_heap'::regclass, - [^;]*'relallvisible',)\s'\d+'::integer - /$1 ''::integer/mgx; - } - - # dumps from pre-9.6 dblink may include redundant ACL settings - if ($old_version lt '9.6') - { - my $comment = - "-- Name: FUNCTION dblink_connect_u\(.*?\); Type: ACL; Schema: public; Owner: .*"; - my $sql = - "REVOKE ALL ON FUNCTION public\.dblink_connect_u\(.*?\) FROM PUBLIC;"; - $dump =~ s/^--\n$comment\n--\n+$sql\n+//mg; - } - - if ($old_version lt '9.3') - { - # CREATE VIEW/RULE statements were not pretty-printed before 9.3. - # To cope, reduce all whitespace sequences within them to one space. - # This must be done on both old and new dumps. - $dump = _mash_view_whitespace($dump); - } - # Suppress blank lines, as some places in pg_dump emit more or fewer. $dump =~ s/\n\n+/\n/g; -- 2.50.1 (Apple Git-155) --WhtY6+QncUkBAPlR-- ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-07-08 21:39 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-04-03 08:20 Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]> 2025-04-03 09:14 ` Ashutosh Bapat <[email protected]> 2025-04-03 13:40 ` Andres Freund <[email protected]> 2026-07-08 21:39 [PATCH v1 1/1] clean up AdjustUpgrade.pm after pg_upgrade min version bump Nathan Bossart <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox