public inbox for [email protected]help / color / mirror / Atom feed
Re: New Object Access Type hooks 8+ messages / 5 participants [nested] [flat]
* Re: New Object Access Type hooks @ 2022-03-22 05:03 Thomas Munro <[email protected]> 0 siblings, 2 replies; 8+ messages in thread From: Thomas Munro @ 2022-03-22 05:03 UTC (permalink / raw) To: Mark Dilger <[email protected]>; +Cc: pgsql-hackers; Joshua Brindle <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; Jeff Davis <[email protected]>; Joe Conway <[email protected]> On Fri, Mar 18, 2022 at 4:22 PM Mark Dilger <[email protected]> wrote: > (FYI, I got a test failure from src/test/recovery/t/013_crash_restart.pl when testing v1-0001. I'm not sure yet what that is about.) Doesn't look like 0001 has anything to do with that... Are you on a Mac? Did it look like this recent failure from CI? https://cirrus-ci.com/task/4686108033286144 https://api.cirrus-ci.com/v1/artifact/task/4686108033286144/log/src/test/recovery/tmp_check/log/regr... https://api.cirrus-ci.com/v1/artifact/task/4686108033286144/log/src/test/recovery/tmp_check/log/013_... I have no idea what is going on there, but searching for discussion brought me here... ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: New Object Access Type hooks @ 2022-03-22 13:25 Andrew Dunstan <[email protected]> parent: Thomas Munro <[email protected]> 1 sibling, 0 replies; 8+ messages in thread From: Andrew Dunstan @ 2022-03-22 13:25 UTC (permalink / raw) To: Thomas Munro <[email protected]>; Mark Dilger <[email protected]>; +Cc: pgsql-hackers; Joshua Brindle <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Jeff Davis <[email protected]>; Joe Conway <[email protected]> On 3/22/22 01:03, Thomas Munro wrote: > On Fri, Mar 18, 2022 at 4:22 PM Mark Dilger > <[email protected]> wrote: >> (FYI, I got a test failure from src/test/recovery/t/013_crash_restart.pl when testing v1-0001. I'm not sure yet what that is about.) > Doesn't look like 0001 has anything to do with that... Are you on a > Mac? Did it look like this recent failure from CI? Probably not connected. It's working fine for me on Ubuntu/ cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: New Object Access Type hooks @ 2022-04-04 16:16 Mark Dilger <[email protected]> parent: Thomas Munro <[email protected]> 1 sibling, 1 reply; 8+ messages in thread From: Mark Dilger @ 2022-04-04 16:16 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: pgsql-hackers; Joshua Brindle <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; Jeff Davis <[email protected]>; Joe Conway <[email protected]> > On Mar 21, 2022, at 10:03 PM, Thomas Munro <[email protected]> wrote: > > On Fri, Mar 18, 2022 at 4:22 PM Mark Dilger > <[email protected]> wrote: >> (FYI, I got a test failure from src/test/recovery/t/013_crash_restart.pl when testing v1-0001. I'm not sure yet what that is about.) > > Doesn't look like 0001 has anything to do with that... Are you on a > Mac? Did it look like this recent failure from CI? > > https://cirrus-ci.com/task/4686108033286144 > https://api.cirrus-ci.com/v1/artifact/task/4686108033286144/log/src/test/recovery/tmp_check/log/regr... > https://api.cirrus-ci.com/v1/artifact/task/4686108033286144/log/src/test/recovery/tmp_check/log/013_... > > I have no idea what is going on there, but searching for discussion > brought me here... I just got a crash in this test again. Are you still interested? I still have the logs. No core file appears to have been generated. The test failure is not ok 5 - psql query died successfully after SIGQUIT — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: New Object Access Type hooks @ 2022-04-04 17:03 Tom Lane <[email protected]> parent: Mark Dilger <[email protected]> 0 siblings, 1 reply; 8+ messages in thread From: Tom Lane @ 2022-04-04 17:03 UTC (permalink / raw) To: Mark Dilger <[email protected]>; +Cc: Thomas Munro <[email protected]>; pgsql-hackers; Joshua Brindle <[email protected]>; Robert Haas <[email protected]>; Andrew Dunstan <[email protected]>; Jeff Davis <[email protected]>; Joe Conway <[email protected]> Mark Dilger <[email protected]> writes: > I just got a crash in this test again. Are you still interested? I still have the logs. No core file appears to have been generated. > The test failure is > not ok 5 - psql query died successfully after SIGQUIT Hmm ... I can see one problem with that test: ok( pump_until( $killme, $psql_timeout, \$killme_stderr, qr/WARNING: terminating connection because of crash of another server process|server closed the connection unexpectedly|connection to server was lost/m ), "psql query died successfully after SIGQUIT"); It's been a little while since that message looked like that. Nowadays you get WARNING: terminating connection because of unexpected SIGQUIT signal server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. Usually the test would succeed anyway because of matching the second or third regex alternative, but I wonder if there is some other spelling of libpq's complaint that shows up occasionally. It'd be nice if we could see the contents of $killme_stderr upon failure. regards, tom lane ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: New Object Access Type hooks @ 2022-04-04 17:41 Tom Lane <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 1 reply; 8+ messages in thread From: Tom Lane @ 2022-04-04 17:41 UTC (permalink / raw) To: Mark Dilger <[email protected]>; +Cc: Thomas Munro <[email protected]>; pgsql-hackers; Joshua Brindle <[email protected]>; Robert Haas <[email protected]>; Andrew Dunstan <[email protected]>; Jeff Davis <[email protected]>; Joe Conway <[email protected]> I wrote: > Usually the test would succeed anyway because of matching the > second or third regex alternative, but I wonder if there is > some other spelling of libpq's complaint that shows up > occasionally. It'd be nice if we could see the contents of > $killme_stderr upon failure. OK, now I'm confused, because pump_until is very clearly *trying* to report exactly that: if (not $proc->pumpable()) { diag("pump_until: process terminated unexpectedly when searching for \"$until\" with stream: \"$$stream\""); return 0; } and if I intentionally break the regex then I do see this output when running the test by hand: # Running: pg_ctl kill QUIT 1922645 ok 4 - killed process with SIGQUIT # pump_until: process terminated unexpectedly when searching for "(?^m:WARNING: terminating connection because of crash of another server process|server closed the connection foounexpectedly|connection to server was lost)" with stream: "psql:<stdin>:9: WARNING: terminating connection because of unexpected SIGQUIT signal # psql:<stdin>:9: server closed the connection unexpectedly # This probably means the server terminated abnormally # before or while processing the request. # " not ok 5 - psql query died successfully after SIGQUIT Is our CI setup failing to capture stderr from TAP tests?? regards, tom lane ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: New Object Access Type hooks @ 2022-04-04 17:44 Mark Dilger <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 1 reply; 8+ messages in thread From: Mark Dilger @ 2022-04-04 17:44 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Thomas Munro <[email protected]>; pgsql-hackers; Joshua Brindle <[email protected]>; Robert Haas <[email protected]>; Andrew Dunstan <[email protected]>; Jeff Davis <[email protected]>; Joe Conway <[email protected]> > On Apr 4, 2022, at 10:41 AM, Tom Lane <[email protected]> wrote: > > Is our CI setup failing to capture stderr from TAP tests?? I'm looking into the way our TAP test infrastructure assigns port numbers to nodes, and whether that is reasonable during parallel test runs with nodes stopping and starting again. On casual inspection, that doesn't seem ok, because the Cluster.pm logic to make sure nodes get unique ports doesn't seem to be thinking about other parallel tests running. It will notice if another node is already bound to the port, but if another node has been killed and not yet restarted, won't things go off the rails? I'm writing a parallel test just for this. Will get back to you. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: New Object Access Type hooks @ 2022-04-04 18:32 Mark Dilger <[email protected]> parent: Mark Dilger <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Mark Dilger @ 2022-04-04 18:32 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Thomas Munro <[email protected]>; pgsql-hackers; Joshua Brindle <[email protected]>; Robert Haas <[email protected]>; Andrew Dunstan <[email protected]>; Jeff Davis <[email protected]>; Joe Conway <[email protected]> > On Apr 4, 2022, at 10:44 AM, Mark Dilger <[email protected]> wrote: > > I'm writing a parallel test just for this. Will get back to you. Ok, that experiment didn't accomplish anything, beyond refreshing my memory regarding Cluster.pm preferring sockets over ports. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 8+ messages in thread
* [PATCH v9 1/4] Change AclMode from a uint32 to a uint64. @ 2022-09-08 05:25 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Nathan Bossart @ 2022-09-08 05:25 UTC (permalink / raw) --- src/backend/nodes/outfuncs.c | 2 +- src/bin/pg_upgrade/check.c | 35 +++++++++++++++++++++++++++++++++ src/include/catalog/pg_type.dat | 4 ++-- src/include/nodes/parsenodes.h | 6 +++--- src/include/utils/acl.h | 28 +++++++++++++------------- 5 files changed, 55 insertions(+), 20 deletions(-) diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index f05e72f0dc..8f150e9a2e 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -560,7 +560,7 @@ _outRangeTblEntry(StringInfo str, const RangeTblEntry *node) WRITE_BOOL_FIELD(lateral); WRITE_BOOL_FIELD(inh); WRITE_BOOL_FIELD(inFromCl); - WRITE_UINT_FIELD(requiredPerms); + WRITE_UINT64_FIELD(requiredPerms); WRITE_OID_FIELD(checkAsUser); WRITE_BITMAPSET_FIELD(selectedCols); WRITE_BITMAPSET_FIELD(insertedCols); diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c index f1bc1e6886..615a53a864 100644 --- a/src/bin/pg_upgrade/check.c +++ b/src/bin/pg_upgrade/check.c @@ -28,6 +28,7 @@ static void check_for_incompatible_polymorphics(ClusterInfo *cluster); static void check_for_tables_with_oids(ClusterInfo *cluster); static void check_for_composite_data_type_usage(ClusterInfo *cluster); static void check_for_reg_data_type_usage(ClusterInfo *cluster); +static void check_for_aclitem_data_type_usage(ClusterInfo *cluster); static void check_for_jsonb_9_4_usage(ClusterInfo *cluster); static void check_for_pg_role_prefix(ClusterInfo *cluster); static void check_for_new_tablespace_dir(ClusterInfo *new_cluster); @@ -107,6 +108,13 @@ check_and_dump_old_cluster(bool live_check) check_for_reg_data_type_usage(&old_cluster); check_for_isn_and_int8_passing_mismatch(&old_cluster); + /* + * PG 16 increased the size of the 'aclitem' type, which breaks the on-disk + * format for existing data. + */ + if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1500) + check_for_aclitem_data_type_usage(&old_cluster); + /* * PG 14 changed the function signature of encoding conversion functions. * Conversions from older versions cannot be upgraded automatically @@ -1319,6 +1327,33 @@ check_for_reg_data_type_usage(ClusterInfo *cluster) check_ok(); } +/* + * check_for_aclitem_data_type_usage + * + * aclitem changed its storage format in 16, so check for it. + */ +static void +check_for_aclitem_data_type_usage(ClusterInfo *cluster) +{ + char output_path[MAXPGPATH]; + + prep_status("Checking for incompatible aclitem data type in user tables"); + + snprintf(output_path, sizeof(output_path), "tables_using_aclitem.txt"); + + if (check_for_data_type_usage(cluster, "pg_catalog.aclitem", output_path)) + { + pg_log(PG_REPORT, "fatal"); + pg_fatal("Your installation contains the \"aclitem\" data type in user tables.\n" + "The internal format of \"aclitem\" changed in PostgreSQL version 16\n" + "so this cluster cannot currently be upgraded. You can drop the\n" + "problem columns and restart the upgrade. A list of the problem\n" + "columns is in the file:\n" + " %s", output_path); + } + else + check_ok(); +} /* * check_for_jsonb_9_4_usage() diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat index df45879463..0763dfde39 100644 --- a/src/include/catalog/pg_type.dat +++ b/src/include/catalog/pg_type.dat @@ -267,9 +267,9 @@ # OIDS 1000 - 1099 { oid => '1033', array_type_oid => '1034', descr => 'access control list', - typname => 'aclitem', typlen => '12', typbyval => 'f', typcategory => 'U', + typname => 'aclitem', typlen => '16', typbyval => 'f', typcategory => 'U', typinput => 'aclitemin', typoutput => 'aclitemout', typreceive => '-', - typsend => '-', typalign => 'i' }, + typsend => '-', typalign => 'd' }, { oid => '1042', array_type_oid => '1014', descr => 'char(length), blank-padded string, fixed storage length', typname => 'bpchar', typlen => '-1', typbyval => 'f', typcategory => 'S', diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 7caff62af7..f4ed9bbff9 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -73,12 +73,12 @@ typedef enum SetQuantifier /* * Grantable rights are encoded so that we can OR them together in a bitmask. - * The present representation of AclItem limits us to 16 distinct rights, - * even though AclMode is defined as uint32. See utils/acl.h. + * The present representation of AclItem limits us to 32 distinct rights, + * even though AclMode is defined as uint64. See utils/acl.h. * * Caution: changing these codes breaks stored ACLs, hence forces initdb. */ -typedef uint32 AclMode; /* a bitmask of privilege bits */ +typedef uint64 AclMode; /* a bitmask of privilege bits */ #define ACL_INSERT (1<<0) /* for relations */ #define ACL_SELECT (1<<1) diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h index 35b3d8dd88..f920dd206f 100644 --- a/src/include/utils/acl.h +++ b/src/include/utils/acl.h @@ -59,33 +59,33 @@ typedef struct AclItem } AclItem; /* - * The upper 16 bits of the ai_privs field of an AclItem are the grant option - * bits, and the lower 16 bits are the actual privileges. We use "rights" + * The upper 32 bits of the ai_privs field of an AclItem are the grant option + * bits, and the lower 32 bits are the actual privileges. We use "rights" * to mean the combined grant option and privilege bits fields. */ -#define ACLITEM_GET_PRIVS(item) ((item).ai_privs & 0xFFFF) -#define ACLITEM_GET_GOPTIONS(item) (((item).ai_privs >> 16) & 0xFFFF) +#define ACLITEM_GET_PRIVS(item) ((item).ai_privs & 0xFFFFFFFF) +#define ACLITEM_GET_GOPTIONS(item) (((item).ai_privs >> 32) & 0xFFFFFFFF) #define ACLITEM_GET_RIGHTS(item) ((item).ai_privs) -#define ACL_GRANT_OPTION_FOR(privs) (((AclMode) (privs) & 0xFFFF) << 16) -#define ACL_OPTION_TO_PRIVS(privs) (((AclMode) (privs) >> 16) & 0xFFFF) +#define ACL_GRANT_OPTION_FOR(privs) (((AclMode) (privs) & 0xFFFFFFFF) << 32) +#define ACL_OPTION_TO_PRIVS(privs) (((AclMode) (privs) >> 32) & 0xFFFFFFFF) #define ACLITEM_SET_PRIVS(item,privs) \ - ((item).ai_privs = ((item).ai_privs & ~((AclMode) 0xFFFF)) | \ - ((AclMode) (privs) & 0xFFFF)) + ((item).ai_privs = ((item).ai_privs & ~((AclMode) 0xFFFFFFFF)) | \ + ((AclMode) (privs) & 0xFFFFFFFF)) #define ACLITEM_SET_GOPTIONS(item,goptions) \ - ((item).ai_privs = ((item).ai_privs & ~(((AclMode) 0xFFFF) << 16)) | \ - (((AclMode) (goptions) & 0xFFFF) << 16)) + ((item).ai_privs = ((item).ai_privs & ~(((AclMode) 0xFFFFFFFF) << 32)) | \ + (((AclMode) (goptions) & 0xFFFFFFFF) << 32)) #define ACLITEM_SET_RIGHTS(item,rights) \ ((item).ai_privs = (AclMode) (rights)) #define ACLITEM_SET_PRIVS_GOPTIONS(item,privs,goptions) \ - ((item).ai_privs = ((AclMode) (privs) & 0xFFFF) | \ - (((AclMode) (goptions) & 0xFFFF) << 16)) + ((item).ai_privs = ((AclMode) (privs) & 0xFFFFFFFF) | \ + (((AclMode) (goptions) & 0xFFFFFFFF) << 32)) -#define ACLITEM_ALL_PRIV_BITS ((AclMode) 0xFFFF) -#define ACLITEM_ALL_GOPTION_BITS ((AclMode) 0xFFFF << 16) +#define ACLITEM_ALL_PRIV_BITS ((AclMode) 0xFFFFFFFF) +#define ACLITEM_ALL_GOPTION_BITS ((AclMode) 0xFFFFFFFF << 32) /* * Definitions for convenient access to Acl (array of AclItem). -- 2.25.1 --AhhlLboLdkugWU4S Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v9-0002-Simplify-WARNING-messages-emitted-when-skipping-v.patch" ^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2022-09-08 05:25 UTC | newest] Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-03-22 05:03 Re: New Object Access Type hooks Thomas Munro <[email protected]> 2022-03-22 13:25 ` Andrew Dunstan <[email protected]> 2022-04-04 16:16 ` Mark Dilger <[email protected]> 2022-04-04 17:03 ` Tom Lane <[email protected]> 2022-04-04 17:41 ` Tom Lane <[email protected]> 2022-04-04 17:44 ` Mark Dilger <[email protected]> 2022-04-04 18:32 ` Mark Dilger <[email protected]> 2022-09-08 05:25 [PATCH v9 1/4] Change AclMode from a uint32 to a uint64. Nathan Bossart <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox