agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v20 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. 4+ messages / 2 participants [nested] [flat]
* [PATCH v20 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index eead88419f..3c419672fc 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25843,16 +25843,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 3324ffe16a..8390626e49 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 186ced4a35..381f2e196f 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10911,13 +10911,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --Z1Z8UV8BNhgCynIS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v20-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 4+ messages in thread
* [PATCH v68 19/31] pgstat: test: test stats interactions with streaming replication. @ 2022-03-21 19:58 Andres Freund <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Andres Freund @ 2022-03-21 19:58 UTC (permalink / raw) Author: Melanie Plageman <[email protected]> Discussion: https://postgr.es/m/[email protected] --- .../recovery/t/030_stats_cleanup_replica.pl | 228 ++++++++++++++++++ 1 file changed, 228 insertions(+) create mode 100644 src/test/recovery/t/030_stats_cleanup_replica.pl diff --git a/src/test/recovery/t/030_stats_cleanup_replica.pl b/src/test/recovery/t/030_stats_cleanup_replica.pl new file mode 100644 index 00000000000..ea79e990395 --- /dev/null +++ b/src/test/recovery/t/030_stats_cleanup_replica.pl @@ -0,0 +1,228 @@ +# Copyright (c) 2021-2022, PostgreSQL Global Development Group + +# Tests that statistics are removed from a physical replica after being dropped +# on the primary + +use strict; +use warnings; +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +# Initialize primary node +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +# A specific role is created to perform some tests related to replication, +# and it needs proper authentication configuration. +$node_primary->init( + allows_streaming => 1, + auth_extra => [ '--create-role', 'repl_role' ]); +$node_primary->start; +my $backup_name = 'my_backup'; + +# Set track_functions to all on primary +$node_primary->append_conf('postgresql.conf', "track_functions = 'all'"); +$node_primary->reload; + +# Take backup +$node_primary->backup($backup_name); + +# Create streaming standby linking to primary +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +$node_standby->init_from_backup($node_primary, $backup_name, + has_streaming => 1); +$node_standby->start; + +sub populate_standby_stats +{ + my ($node_primary, $node_standby, $connect_db, $schema) = @_; + # Create table on primary + $node_primary->safe_psql($connect_db, + "CREATE TABLE $schema.drop_tab_test1 AS SELECT generate_series(1,100) AS a" + ); + + # Create function on primary + $node_primary->safe_psql($connect_db, + "CREATE FUNCTION $schema.drop_func_test1() RETURNS VOID AS 'select 2;' LANGUAGE SQL IMMUTABLE" + ); + + # Wait for catchup + my $primary_lsn = $node_primary->lsn('write'); + $node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + + # Get database oid + my $dboid = $node_standby->safe_psql($connect_db, + "SELECT oid FROM pg_database WHERE datname = '$connect_db'"); + + # Get table oid + my $tableoid = $node_standby->safe_psql($connect_db, + "SELECT '$schema.drop_tab_test1'::regclass::oid"); + + # Do scan on standby + $node_standby->safe_psql($connect_db, + "SELECT * FROM $schema.drop_tab_test1"); + + # Get function oid + my $funcoid = $node_standby->safe_psql($connect_db, + "SELECT '$schema.drop_func_test1()'::regprocedure::oid"); + + # Call function on standby + $node_standby->safe_psql($connect_db, "SELECT $schema.drop_func_test1()"); + + # Force flush of stats on standby + $node_standby->safe_psql($connect_db, + "SELECT pg_stat_force_next_flush()"); + + return ($dboid, $tableoid, $funcoid); +} + +sub drop_function_by_oid +{ + my ($node_primary, $connect_db, $funcoid) = @_; + + # Get function name from returned oid + my $func_name = $node_primary->safe_psql($connect_db, + "SELECT '$funcoid'::regprocedure"); + + # Drop function on primary + $node_primary->safe_psql($connect_db, "DROP FUNCTION $func_name"); +} + +sub drop_table_by_oid +{ + my ($node_primary, $connect_db, $tableoid) = @_; + + # Get table name from returned oid + my $table_name = + $node_primary->safe_psql($connect_db, "SELECT '$tableoid'::regclass"); + + # Drop table on primary + $node_primary->safe_psql($connect_db, "DROP TABLE $table_name"); +} + +sub test_standby_func_tab_stats_status +{ + local $Test::Builder::Level = $Test::Builder::Level + 1; + my ($node_standby, $connect_db, $dboid, $tableoid, $funcoid, $present) = + @_; + + is( $node_standby->safe_psql( + $connect_db, + "SELECT pg_stat_exists_stat('relation', $dboid, $tableoid)"), + $present, + "Check that table stats exist is '$present' on standby"); + + is( $node_standby->safe_psql( + $connect_db, + "SELECT pg_stat_exists_stat('function', $dboid, $funcoid)"), + $present, + "Check that function stats exist is '$present' on standby"); + + return; +} + +sub test_standby_db_stats_status +{ + local $Test::Builder::Level = $Test::Builder::Level + 1; + my ($node_standby, $connect_db, $dboid, $present) = @_; + + is( $node_standby->safe_psql( + $connect_db, "SELECT pg_stat_exists_stat('database', $dboid, 0)"), + $present, + "Check that db stats exist is '$present' on standby"); +} + +# Test that stats are cleaned up on standby after dropping table or function + +# Populate test objects +my ($dboid, $tableoid, $funcoid) = + populate_standby_stats($node_primary, $node_standby, 'postgres', 'public'); + +# Test that the stats are present +test_standby_func_tab_stats_status($node_standby, 'postgres', + $dboid, $tableoid, $funcoid, 't'); + +# Drop test objects +drop_table_by_oid($node_primary, 'postgres', $tableoid); + +drop_function_by_oid($node_primary, 'postgres', $funcoid); + +# Wait for catchup +my $primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Force flush of stats on standby +$node_standby->safe_psql('postgres', "SELECT pg_stat_force_next_flush()"); + +# Check table and function stats removed from standby +test_standby_func_tab_stats_status($node_standby, 'postgres', + $dboid, $tableoid, $funcoid, 'f'); + +# Check that stats are cleaned up on standby after dropping schema + +# Create schema +$node_primary->safe_psql('postgres', "CREATE SCHEMA drop_schema_test1"); + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Populate test objects +($dboid, $tableoid, $funcoid) = populate_standby_stats($node_primary, + $node_standby, 'postgres', 'drop_schema_test1'); + +# Test that the stats are present +test_standby_func_tab_stats_status($node_standby, 'postgres', + $dboid, $tableoid, $funcoid, 't'); + +# Drop schema +$node_primary->safe_psql('postgres', "DROP SCHEMA drop_schema_test1 CASCADE"); + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Force flush of stats on standby +$node_standby->safe_psql('postgres', "SELECT pg_stat_force_next_flush()"); + +# Check table and function stats removed from standby +test_standby_func_tab_stats_status($node_standby, 'postgres', + $dboid, $tableoid, $funcoid, 'f'); + +# Test that stats are cleaned up on standby after dropping database + +# Create the database +$node_primary->safe_psql('postgres', "CREATE DATABASE test"); + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Populate test objects +($dboid, $tableoid, $funcoid) = + populate_standby_stats($node_primary, $node_standby, 'test', 'public'); + +# Test that the stats are present +test_standby_func_tab_stats_status($node_standby, 'test', + $dboid, $tableoid, $funcoid, 't'); + +test_standby_db_stats_status($node_standby, 'test', $dboid, 't'); + +# Drop db 'test' on primary +$node_primary->safe_psql('postgres', "DROP DATABASE test"); + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Force flush of stats on standby +$node_standby->safe_psql('postgres', "SELECT pg_stat_force_next_flush()"); + +# Test that the stats were cleaned up on standby +# Note that this connects to 'postgres' but provides the dboid of dropped db +# 'test' which was returned by previous routine +test_standby_func_tab_stats_status($node_standby, 'postgres', + $dboid, $tableoid, $funcoid, 'f'); + +test_standby_db_stats_status($node_standby, 'postgres', $dboid, 'f'); + +done_testing(); -- 2.35.1.677.gabf474a5dd --c4n45bccvxafocvc Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v68-0020-pgstat-test-stats-handling-of-restarts-including.patch" ^ permalink raw reply [nested|flat] 4+ messages in thread
* [PATCH v69 15/28] pgstat: test: test stats interactions with streaming replication. @ 2022-03-21 19:58 Andres Freund <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Andres Freund @ 2022-03-21 19:58 UTC (permalink / raw) Author: Melanie Plageman <[email protected]> Discussion: https://postgr.es/m/[email protected] --- .../recovery/t/030_stats_cleanup_replica.pl | 218 ++++++++++++++++++ 1 file changed, 218 insertions(+) create mode 100644 src/test/recovery/t/030_stats_cleanup_replica.pl diff --git a/src/test/recovery/t/030_stats_cleanup_replica.pl b/src/test/recovery/t/030_stats_cleanup_replica.pl new file mode 100644 index 00000000000..be7e62a9cb4 --- /dev/null +++ b/src/test/recovery/t/030_stats_cleanup_replica.pl @@ -0,0 +1,218 @@ +# Copyright (c) 2021-2022, PostgreSQL Global Development Group + +# Tests that statistics are removed from a physical replica after being dropped +# on the primary + +use strict; +use warnings; +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +# Initialize primary node +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +# A specific role is created to perform some tests related to replication, +# and it needs proper authentication configuration. +$node_primary->init(allows_streaming => 1); + +# Set track_functions to all on primary +$node_primary->append_conf('postgresql.conf', "track_functions = 'all'"); +$node_primary->start; + +# Take backup +my $backup_name = 'my_backup'; +$node_primary->backup($backup_name); + +# Create streaming standby linking to primary +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +$node_standby->init_from_backup($node_primary, $backup_name, + has_streaming => 1); +$node_standby->start; + +my $sect = 'initial'; + +sub populate_standby_stats +{ + my ($connect_db, $schema) = @_; + # Create table on primary + $node_primary->safe_psql($connect_db, + "CREATE TABLE $schema.drop_tab_test1 AS SELECT generate_series(1,100) AS a" + ); + + # Create function on primary + $node_primary->safe_psql($connect_db, + "CREATE FUNCTION $schema.drop_func_test1() RETURNS VOID AS 'select 2;' LANGUAGE SQL IMMUTABLE" + ); + + # Wait for catchup + my $primary_lsn = $node_primary->lsn('write'); + $node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + + # Get database oid + my $dboid = $node_standby->safe_psql($connect_db, + "SELECT oid FROM pg_database WHERE datname = '$connect_db'"); + + # Get table oid + my $tableoid = $node_standby->safe_psql($connect_db, + "SELECT '$schema.drop_tab_test1'::regclass::oid"); + + # Do scan on standby + $node_standby->safe_psql($connect_db, + "SELECT * FROM $schema.drop_tab_test1"); + + # Get function oid + my $funcoid = $node_standby->safe_psql($connect_db, + "SELECT '$schema.drop_func_test1()'::regprocedure::oid"); + + # Call function on standby + $node_standby->safe_psql($connect_db, "SELECT $schema.drop_func_test1()"); + + return ($dboid, $tableoid, $funcoid); +} + +sub drop_function_by_oid +{ + my ($connect_db, $funcoid) = @_; + + # Get function name from returned oid + my $func_name = $node_primary->safe_psql($connect_db, + "SELECT '$funcoid'::regprocedure"); + + # Drop function on primary + $node_primary->safe_psql($connect_db, "DROP FUNCTION $func_name"); +} + +sub drop_table_by_oid +{ + my ($connect_db, $tableoid) = @_; + + # Get table name from returned oid + my $table_name = + $node_primary->safe_psql($connect_db, "SELECT '$tableoid'::regclass"); + + # Drop table on primary + $node_primary->safe_psql($connect_db, "DROP TABLE $table_name"); +} + +sub test_standby_func_tab_stats_status +{ + local $Test::Builder::Level = $Test::Builder::Level + 1; + my ($connect_db, $dboid, $tableoid, $funcoid, $present) = @_; + + my %expected = (rel => $present, func => $present); + my %stats; + + $stats{rel} = $node_standby->safe_psql($connect_db, + "SELECT pg_stat_exists_stat('relation', $dboid, $tableoid)"); + $stats{func} = $node_standby->safe_psql($connect_db, + "SELECT pg_stat_exists_stat('function', $dboid, $funcoid)"); + + is_deeply(\%stats, \%expected, "$sect: standby stats as expected"); + + return; +} + +sub test_standby_db_stats_status +{ + local $Test::Builder::Level = $Test::Builder::Level + 1; + my ($connect_db, $dboid, $present) = @_; + + is( $node_standby->safe_psql( + $connect_db, "SELECT pg_stat_exists_stat('database', $dboid, 0)"), + $present, + "$sect: standby db stats as expected"); +} + +# Test that stats are cleaned up on standby after dropping table or function + +# Populate test objects +my ($dboid, $tableoid, $funcoid) = + populate_standby_stats('postgres', 'public'); + +# Test that the stats are present +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 't'); + +# Drop test objects +drop_table_by_oid('postgres', $tableoid); + +drop_function_by_oid('postgres', $funcoid); + +$sect = 'post drop'; + +# Wait for catchup +my $primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Check table and function stats removed from standby +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 'f'); + +# Check that stats are cleaned up on standby after dropping schema + +$sect = "schema creation"; + +# Create schema +$node_primary->safe_psql('postgres', "CREATE SCHEMA drop_schema_test1"); + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Populate test objects +($dboid, $tableoid, $funcoid) = + populate_standby_stats('postgres', 'drop_schema_test1'); + +# Test that the stats are present +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 't'); + +# Drop schema +$node_primary->safe_psql('postgres', "DROP SCHEMA drop_schema_test1 CASCADE"); + +$sect = "post schema drop"; + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Check table and function stats removed from standby +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 'f'); + +# Test that stats are cleaned up on standby after dropping database + +# Create the database +$node_primary->safe_psql('postgres', "CREATE DATABASE test"); + +$sect = "createdb"; + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Populate test objects +($dboid, $tableoid, $funcoid) = populate_standby_stats('test', 'public'); + +# Test that the stats are present +test_standby_func_tab_stats_status('test', $dboid, $tableoid, $funcoid, 't'); + +test_standby_db_stats_status('test', $dboid, 't'); + +# Drop db 'test' on primary +$node_primary->safe_psql('postgres', "DROP DATABASE test"); +$sect = "post dropdb"; + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Test that the stats were cleaned up on standby +# Note that this connects to 'postgres' but provides the dboid of dropped db +# 'test' which was returned by previous routine +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 'f'); + +test_standby_db_stats_status('postgres', $dboid, 'f'); + +done_testing(); -- 2.35.1.677.gabf474a5dd --be3jiks7ge4r32o3 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v69-0016-pgstat-test-stats-handling-of-restarts-including.patch" ^ permalink raw reply [nested|flat] 4+ messages in thread
* [PATCH v70 20/27] pgstat: test: test stats interactions with streaming replication. @ 2022-03-21 19:58 Andres Freund <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Andres Freund @ 2022-03-21 19:58 UTC (permalink / raw) Author: Melanie Plageman <[email protected]> Discussion: https://postgr.es/m/[email protected] --- .../recovery/t/030_stats_cleanup_replica.pl | 243 ++++++++++++++++++ 1 file changed, 243 insertions(+) create mode 100644 src/test/recovery/t/030_stats_cleanup_replica.pl diff --git a/src/test/recovery/t/030_stats_cleanup_replica.pl b/src/test/recovery/t/030_stats_cleanup_replica.pl new file mode 100644 index 00000000000..6b8998e5da9 --- /dev/null +++ b/src/test/recovery/t/030_stats_cleanup_replica.pl @@ -0,0 +1,243 @@ +# Copyright (c) 2021-2022, PostgreSQL Global Development Group + +# Tests that statistics are removed from a physical replica after being dropped +# on the primary + +use strict; +use warnings; +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +# Initialize primary node +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +# A specific role is created to perform some tests related to replication, +# and it needs proper authentication configuration. +$node_primary->init(allows_streaming => 1); + +# Set track_functions to all on primary +$node_primary->append_conf('postgresql.conf', "track_functions = 'all'"); +$node_primary->start; + +# Take backup +my $backup_name = 'my_backup'; +$node_primary->backup($backup_name); + +# Create streaming standby linking to primary +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +$node_standby->init_from_backup($node_primary, $backup_name, + has_streaming => 1); +$node_standby->start; + +my $sect = 'initial'; + +sub populate_standby_stats +{ + my ($connect_db, $schema) = @_; + # Create table on primary + $node_primary->safe_psql($connect_db, + "CREATE TABLE $schema.drop_tab_test1 AS SELECT generate_series(1,100) AS a" + ); + + # Create function on primary + $node_primary->safe_psql($connect_db, + "CREATE FUNCTION $schema.drop_func_test1() RETURNS VOID AS 'select 2;' LANGUAGE SQL IMMUTABLE" + ); + + # Wait for catchup + my $primary_lsn = $node_primary->lsn('write'); + $node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + + # Get database oid + my $dboid = $node_standby->safe_psql($connect_db, + "SELECT oid FROM pg_database WHERE datname = '$connect_db'"); + + # Get table oid + my $tableoid = $node_standby->safe_psql($connect_db, + "SELECT '$schema.drop_tab_test1'::regclass::oid"); + + # Do scan on standby + $node_standby->safe_psql($connect_db, + "SELECT * FROM $schema.drop_tab_test1"); + + # Get function oid + my $funcoid = $node_standby->safe_psql($connect_db, + "SELECT '$schema.drop_func_test1()'::regprocedure::oid"); + + # Call function on standby + $node_standby->safe_psql($connect_db, "SELECT $schema.drop_func_test1()"); + + return ($dboid, $tableoid, $funcoid); +} + +sub drop_function_by_oid +{ + my ($connect_db, $funcoid) = @_; + + # Get function name from returned oid + my $func_name = $node_primary->safe_psql($connect_db, + "SELECT '$funcoid'::regprocedure"); + + # Drop function on primary + $node_primary->safe_psql($connect_db, "DROP FUNCTION $func_name"); +} + +sub drop_table_by_oid +{ + my ($connect_db, $tableoid) = @_; + + # Get table name from returned oid + my $table_name = + $node_primary->safe_psql($connect_db, "SELECT '$tableoid'::regclass"); + + # Drop table on primary + $node_primary->safe_psql($connect_db, "DROP TABLE $table_name"); +} + +sub test_standby_func_tab_stats_status +{ + local $Test::Builder::Level = $Test::Builder::Level + 1; + my ($connect_db, $dboid, $tableoid, $funcoid, $present) = @_; + + my %expected = (rel => $present, func => $present); + my %stats; + + $stats{rel} = $node_standby->safe_psql($connect_db, + "SELECT pg_stat_exists_stat('relation', $dboid, $tableoid)"); + $stats{func} = $node_standby->safe_psql($connect_db, + "SELECT pg_stat_exists_stat('function', $dboid, $funcoid)"); + + is_deeply(\%stats, \%expected, "$sect: standby stats as expected"); + + return; +} + +sub test_standby_db_stats_status +{ + local $Test::Builder::Level = $Test::Builder::Level + 1; + my ($connect_db, $dboid, $present) = @_; + + is( $node_standby->safe_psql( + $connect_db, "SELECT pg_stat_exists_stat('database', $dboid, 0)"), + $present, + "$sect: standby db stats as expected"); +} + +# Test that stats are cleaned up on standby after dropping table or function + +# Populate test objects +my ($dboid, $tableoid, $funcoid) = + populate_standby_stats('postgres', 'public'); + +# Test that the stats are present +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 't'); + +# Drop test objects +drop_table_by_oid('postgres', $tableoid); + +drop_function_by_oid('postgres', $funcoid); + +$sect = 'post drop'; + +# Wait for catchup +my $primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Check table and function stats removed from standby +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 'f'); + +# Check that stats are cleaned up on standby after dropping schema + +$sect = "schema creation"; + +# Create schema +$node_primary->safe_psql('postgres', "CREATE SCHEMA drop_schema_test1"); + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Populate test objects +($dboid, $tableoid, $funcoid) = + populate_standby_stats('postgres', 'drop_schema_test1'); + +# Test that the stats are present +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 't'); + +# Drop schema +$node_primary->safe_psql('postgres', "DROP SCHEMA drop_schema_test1 CASCADE"); + +$sect = "post schema drop"; + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Check table and function stats removed from standby +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 'f'); + +# Test that stats are cleaned up on standby after dropping database + +# Create the database +$node_primary->safe_psql('postgres', "CREATE DATABASE test"); + +$sect = "createdb"; + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Populate test objects +($dboid, $tableoid, $funcoid) = populate_standby_stats('test', 'public'); + +# Test that the stats are present +test_standby_func_tab_stats_status('test', $dboid, $tableoid, $funcoid, 't'); + +test_standby_db_stats_status('test', $dboid, 't'); + +# Drop db 'test' on primary +$node_primary->safe_psql('postgres', "DROP DATABASE test"); +$sect = "post dropdb"; + +# Wait for catchup +$primary_lsn = $node_primary->lsn('write'); +$node_primary->wait_for_catchup($node_standby, 'replay', $primary_lsn); + +# Test that the stats were cleaned up on standby +# Note that this connects to 'postgres' but provides the dboid of dropped db +# 'test' which was returned by previous routine +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 'f'); + +test_standby_db_stats_status('postgres', $dboid, 'f'); + +# verify that stats persist across graceful restarts on a replica + +# NB: Can't test database stats, they're immediately repopulated when +# reconnecting... +$sect = "pre restart"; +($dboid, $tableoid, $funcoid) = populate_standby_stats('postgres', 'public'); +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 't'); + +$node_standby->restart(); + +$sect = "post non-immediate"; + +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 't'); + +# but gone after an immediate restart +$node_standby->stop('immediate'); +$node_standby->start(); + +$sect = "post immediate restart"; + +test_standby_func_tab_stats_status('postgres', + $dboid, $tableoid, $funcoid, 'f'); + +done_testing(); -- 2.35.1.677.gabf474a5dd --vl7on5vfmrsyxmos Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v70-0021-pgstat-test-subscriber-stats-reset-and-drop.patch" ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2022-03-21 19:58 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-03-09 03:57 [PATCH v20 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2022-03-21 19:58 [PATCH v68 19/31] pgstat: test: test stats interactions with streaming replication. Andres Freund <[email protected]> 2022-03-21 19:58 [PATCH v69 15/28] pgstat: test: test stats interactions with streaming replication. Andres Freund <[email protected]> 2022-03-21 19:58 [PATCH v70 20/27] pgstat: test: test stats interactions with streaming replication. Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox