agora inbox for [email protected]help / color / mirror / Atom feed
64-bit hash function for hstore and citext data type 35+ messages / 10 participants [nested] [flat]
* 64-bit hash function for hstore and citext data type @ 2018-09-26 10:20 amul sul <[email protected]> 0 siblings, 2 replies; 35+ messages in thread From: amul sul @ 2018-09-26 10:20 UTC (permalink / raw) To: pgsql-hackers Hi all, Commit[1] has added 64-bit hash functions for core data types and in the same discussion thread[2] Robert Haas suggested to have the similar extended hash function for hstore and citext data type. Attaching patch proposes the same. Regards, Amul 1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=81c5e46c490e2426db243eada186995da... 2] http://postgr.es/m/CA+Tgmoafx2yoJuhCQQOL5CocEi-w_uG4S2xT0EtgiJnPGcHW3g@mail.gmail.com Attachments: [application/octet-stream] hstore-add-extended-hash-function-v1.patch (5.8K, ../../CAAJ_b947JjnNr9Cp45iNjSqKf6PA5mCTmKsRwPjows93YwQrmw@mail.gmail.com/2-hstore-add-extended-hash-function-v1.patch) download | inline diff: From 2a3b3f96df7f52441ad13e574b8ce3f42b71740b Mon Sep 17 00:00:00 2001 From: Amul Sul <[email protected]> Date: Wed, 26 Sep 2018 05:00:36 -0400 Subject: [PATCH 1/2] hstore - add extended hash function v1 --- contrib/hstore/Makefile | 3 ++- contrib/hstore/expected/hstore.out | 12 ++++++++++++ contrib/hstore/hstore--1.5--1.6.sql | 12 ++++++++++++ contrib/hstore/hstore--unpackaged--1.0.sql | 1 + contrib/hstore/hstore.control | 2 +- contrib/hstore/hstore_op.c | 20 ++++++++++++++++++++ contrib/hstore/sql/hstore.sql | 9 +++++++++ 7 files changed, 57 insertions(+), 2 deletions(-) create mode 100644 contrib/hstore/hstore--1.5--1.6.sql diff --git a/contrib/hstore/Makefile b/contrib/hstore/Makefile index 46d26f8052..8170e307fe 100644 --- a/contrib/hstore/Makefile +++ b/contrib/hstore/Makefile @@ -5,7 +5,8 @@ OBJS = hstore_io.o hstore_op.o hstore_gist.o hstore_gin.o hstore_compat.o \ $(WIN32RES) EXTENSION = hstore -DATA = hstore--1.4.sql hstore--1.4--1.5.sql \ +DATA = hstore--1.4.sql hstore--1.5--1.6.sql \ + hstore--1.4--1.5.sql \ hstore--1.3--1.4.sql hstore--1.2--1.3.sql \ hstore--1.1--1.2.sql hstore--1.0--1.1.sql \ hstore--unpackaged--1.0.sql diff --git a/contrib/hstore/expected/hstore.out b/contrib/hstore/expected/hstore.out index f0d421602d..4f1db01b3e 100644 --- a/contrib/hstore/expected/hstore.out +++ b/contrib/hstore/expected/hstore.out @@ -1515,3 +1515,15 @@ select json_agg(q) from (select f1, hstore_to_json_loose(f2) as f2 from test_jso {"f1":"rec2","f2":{"b": false, "c": "null", "d": -12345, "e": "012345.6", "f": -1.234, "g": 0.345e-4, "a key": 2}}] (1 row) +-- Check the hstore_hash() and hstore_hash_extended() function explicitly. +SELECT v as value, hstore_hash(v)::bit(32) as standard, + hstore_hash_extended(v, 0)::bit(32) as extended0, + hstore_hash_extended(v, 1)::bit(32) as extended1 +FROM (VALUES (NULL::hstore), (''), ('"a key" =>1'), ('c => null'), + ('e => 012345'), ('g => 2.345e+4')) x(v) +WHERE hstore_hash(v)::bit(32) != hstore_hash_extended(v, 0)::bit(32) + OR hstore_hash(v)::bit(32) = hstore_hash_extended(v, 1)::bit(32); + value | standard | extended0 | extended1 +-------+----------+-----------+----------- +(0 rows) + diff --git a/contrib/hstore/hstore--1.5--1.6.sql b/contrib/hstore/hstore--1.5--1.6.sql new file mode 100644 index 0000000000..c5a2bae02f --- /dev/null +++ b/contrib/hstore/hstore--1.5--1.6.sql @@ -0,0 +1,12 @@ +/* contrib/hstore/hstore--1.5--1.6.sql */ + +-- complain if script is sourced in psql, rather than via ALTER EXTENSION +\echo Use "ALTER EXTENSION hstore UPDATE TO '1.6'" to load this file. \quit + +CREATE FUNCTION hstore_hash_extended(hstore, int8) +RETURNS int8 +AS 'MODULE_PATHNAME','hstore_hash_extended' +LANGUAGE C STRICT IMMUTABLE PARALLEL SAFE; + +ALTER OPERATOR FAMILY hash_hstore_ops USING hash ADD + FUNCTION 2 hstore_hash_extended(hstore, int8); diff --git a/contrib/hstore/hstore--unpackaged--1.0.sql b/contrib/hstore/hstore--unpackaged--1.0.sql index 19a7802805..2128165433 100644 --- a/contrib/hstore/hstore--unpackaged--1.0.sql +++ b/contrib/hstore/hstore--unpackaged--1.0.sql @@ -71,6 +71,7 @@ ALTER EXTENSION hstore ADD operator #<=#(hstore,hstore); ALTER EXTENSION hstore ADD operator family btree_hstore_ops using btree; ALTER EXTENSION hstore ADD operator class btree_hstore_ops using btree; ALTER EXTENSION hstore ADD function hstore_hash(hstore); +ALTER EXTENSION hstore ADD function hstore_hash_extended(hstore,int8); ALTER EXTENSION hstore ADD operator family hash_hstore_ops using hash; ALTER EXTENSION hstore ADD operator class hash_hstore_ops using hash; ALTER EXTENSION hstore ADD type ghstore; diff --git a/contrib/hstore/hstore.control b/contrib/hstore/hstore.control index 8a719475b8..93688cdd83 100644 --- a/contrib/hstore/hstore.control +++ b/contrib/hstore/hstore.control @@ -1,5 +1,5 @@ # hstore extension comment = 'data type for storing sets of (key, value) pairs' -default_version = '1.5' +default_version = '1.6' module_pathname = '$libdir/hstore' relocatable = true diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c index 8f9277f8da..898324bbe9 100644 --- a/contrib/hstore/hstore_op.c +++ b/contrib/hstore/hstore_op.c @@ -1253,3 +1253,23 @@ hstore_hash(PG_FUNCTION_ARGS) PG_FREE_IF_COPY(hs, 0); PG_RETURN_DATUM(hval); } + +PG_FUNCTION_INFO_V1(hstore_hash_extended); +Datum +hstore_hash_extended(PG_FUNCTION_ARGS) +{ + HStore *hs = PG_GETARG_HSTORE_P(0); + Datum hval = hash_any_extended((unsigned char *) VARDATA(hs), + VARSIZE(hs) - VARHDRSZ, + PG_GETARG_INT64(1)); + + /* Same approach as hstore_hash */ + Assert(VARSIZE(hs) == + (HS_COUNT(hs) != 0 ? + CALCDATASIZE(HS_COUNT(hs), + HSE_ENDPOS(ARRPTR(hs)[2 * HS_COUNT(hs) - 1])) : + HSHRDSIZE)); + + PG_FREE_IF_COPY(hs, 0); + PG_RETURN_DATUM(hval); +} diff --git a/contrib/hstore/sql/hstore.sql b/contrib/hstore/sql/hstore.sql index d64b9f77c7..76ac48b021 100644 --- a/contrib/hstore/sql/hstore.sql +++ b/contrib/hstore/sql/hstore.sql @@ -350,3 +350,12 @@ insert into test_json_agg values ('rec1','"a key" =>1, b => t, c => null, d=> 12 ('rec2','"a key" =>2, b => f, c => "null", d=> -12345, e => 012345.6, f=> -1.234, g=> 0.345e-4'); select json_agg(q) from test_json_agg q; select json_agg(q) from (select f1, hstore_to_json_loose(f2) as f2 from test_json_agg) q; + +-- Check the hstore_hash() and hstore_hash_extended() function explicitly. +SELECT v as value, hstore_hash(v)::bit(32) as standard, + hstore_hash_extended(v, 0)::bit(32) as extended0, + hstore_hash_extended(v, 1)::bit(32) as extended1 +FROM (VALUES (NULL::hstore), (''), ('"a key" =>1'), ('c => null'), + ('e => 012345'), ('g => 2.345e+4')) x(v) +WHERE hstore_hash(v)::bit(32) != hstore_hash_extended(v, 0)::bit(32) + OR hstore_hash(v)::bit(32) = hstore_hash_extended(v, 1)::bit(32); -- 2.18.0 [application/octet-stream] citext-add-extended-hash-function-v1.patch (6.7K, ../../CAAJ_b947JjnNr9Cp45iNjSqKf6PA5mCTmKsRwPjows93YwQrmw@mail.gmail.com/3-citext-add-extended-hash-function-v1.patch) download | inline diff: From 67231de5dcd1d2f896f941497d44ab14d0d5d352 Mon Sep 17 00:00:00 2001 From: Amul Sul <[email protected]> Date: Wed, 26 Sep 2018 05:32:06 -0400 Subject: [PATCH] citext - add extended hash function v1 --- contrib/citext/Makefile | 3 ++- contrib/citext/citext--1.5--1.6.sql | 12 ++++++++++++ contrib/citext/citext--unpackaged--1.0.sql | 1 + contrib/citext/citext.c | 20 ++++++++++++++++++++ contrib/citext/citext.control | 2 +- contrib/citext/expected/citext.out | 12 ++++++++++++ contrib/citext/expected/citext_1.out | 12 ++++++++++++ contrib/citext/sql/citext.sql | 9 +++++++++ 8 files changed, 69 insertions(+), 2 deletions(-) create mode 100644 contrib/citext/citext--1.5--1.6.sql diff --git a/contrib/citext/Makefile b/contrib/citext/Makefile index e32a7de946..144fcdd794 100644 --- a/contrib/citext/Makefile +++ b/contrib/citext/Makefile @@ -3,7 +3,8 @@ MODULES = citext EXTENSION = citext -DATA = citext--1.4.sql citext--1.4--1.5.sql \ +DATA = citext--1.4.sql citext--1.5--1.6.sql \ + citext--1.4--1.5.sql \ citext--1.3--1.4.sql \ citext--1.2--1.3.sql citext--1.1--1.2.sql \ citext--1.0--1.1.sql citext--unpackaged--1.0.sql diff --git a/contrib/citext/citext--1.5--1.6.sql b/contrib/citext/citext--1.5--1.6.sql new file mode 100644 index 0000000000..32268983ae --- /dev/null +++ b/contrib/citext/citext--1.5--1.6.sql @@ -0,0 +1,12 @@ +/* contrib/citext/citext--1.5--1.6.sql */ + +-- complain if script is sourced in psql, rather than via ALTER EXTENSION +\echo Use "ALTER EXTENSION citext UPDATE TO '1.6'" to load this file. \quit + +CREATE FUNCTION citext_hash_extended(citext, int8) +RETURNS int8 +AS 'MODULE_PATHNAME' +LANGUAGE C STRICT IMMUTABLE PARALLEL SAFE; + +ALTER OPERATOR FAMILY citext_ops USING hash ADD + FUNCTION 2 citext_hash_extended(citext, int8); diff --git a/contrib/citext/citext--unpackaged--1.0.sql b/contrib/citext/citext--unpackaged--1.0.sql index ef6d6b0639..9b1161b239 100644 --- a/contrib/citext/citext--unpackaged--1.0.sql +++ b/contrib/citext/citext--unpackaged--1.0.sql @@ -33,6 +33,7 @@ ALTER EXTENSION citext ADD operator <(citext,citext); ALTER EXTENSION citext ADD operator <=(citext,citext); ALTER EXTENSION citext ADD function citext_cmp(citext,citext); ALTER EXTENSION citext ADD function citext_hash(citext); +ALTER EXTENSION citext ADD function citext_hash_extended(citext,int8); ALTER EXTENSION citext ADD operator family citext_ops using btree; ALTER EXTENSION citext ADD operator class citext_ops using btree; ALTER EXTENSION citext ADD operator family citext_ops using hash; diff --git a/contrib/citext/citext.c b/contrib/citext/citext.c index 2c0e48e2bc..24ceeb11fc 100644 --- a/contrib/citext/citext.c +++ b/contrib/citext/citext.c @@ -153,6 +153,26 @@ citext_hash(PG_FUNCTION_ARGS) PG_RETURN_DATUM(result); } +PG_FUNCTION_INFO_V1(citext_hash_extended); + +Datum +citext_hash_extended(PG_FUNCTION_ARGS) +{ + text *txt = PG_GETARG_TEXT_PP(0); + uint64 seed = PG_GETARG_INT64(1); + char *str; + Datum result; + + str = str_tolower(VARDATA_ANY(txt), VARSIZE_ANY_EXHDR(txt), DEFAULT_COLLATION_OID); + result = hash_any_extended((unsigned char *) str, strlen(str), seed); + pfree(str); + + /* Avoid leaking memory for toasted inputs */ + PG_FREE_IF_COPY(txt, 0); + + PG_RETURN_DATUM(result); +} + /* * ================== * OPERATOR FUNCTIONS diff --git a/contrib/citext/citext.control b/contrib/citext/citext.control index 4cd6e09331..a872a3f012 100644 --- a/contrib/citext/citext.control +++ b/contrib/citext/citext.control @@ -1,5 +1,5 @@ # citext extension comment = 'data type for case-insensitive character strings' -default_version = '1.5' +default_version = '1.6' module_pathname = '$libdir/citext' relocatable = true diff --git a/contrib/citext/expected/citext.out b/contrib/citext/expected/citext.out index 99365c57b0..20890db859 100644 --- a/contrib/citext/expected/citext.out +++ b/contrib/citext/expected/citext.out @@ -222,6 +222,18 @@ SELECT citext_cmp('B'::citext, 'a'::citext) > 0 AS true; t (1 row) +-- Check the citext_hash() and citext_hash_extended() function explicitly. +SELECT v as value, citext_hash(v)::bit(32) as standard, + citext_hash_extended(v, 0)::bit(32) as extended0, + citext_hash_extended(v, 1)::bit(32) as extended1 +FROM (VALUES (NULL::citext), ('PostgreSQL'), ('eIpUEtqmY89'), ('AXKEJBTK'), + ('muop28x03'), ('yi3nm0d73')) x(v) +WHERE citext_hash(v)::bit(32) != citext_hash_extended(v, 0)::bit(32) + OR citext_hash(v)::bit(32) = citext_hash_extended(v, 1)::bit(32); + value | standard | extended0 | extended1 +-------+----------+-----------+----------- +(0 rows) + -- Do some tests using a table and index. CREATE TEMP TABLE try ( name citext PRIMARY KEY diff --git a/contrib/citext/expected/citext_1.out b/contrib/citext/expected/citext_1.out index 8aac72e226..755baad8e2 100644 --- a/contrib/citext/expected/citext_1.out +++ b/contrib/citext/expected/citext_1.out @@ -222,6 +222,18 @@ SELECT citext_cmp('B'::citext, 'a'::citext) > 0 AS true; t (1 row) +-- Check the citext_hash() and citext_hash_extended() function explicitly. +SELECT v as value, citext_hash(v)::bit(32) as standard, + citext_hash_extended(v, 0)::bit(32) as extended0, + citext_hash_extended(v, 1)::bit(32) as extended1 +FROM (VALUES (NULL::citext), ('PostgreSQL'), ('eIpUEtqmY89'), ('AXKEJBTK'), + ('muop28x03'), ('yi3nm0d73')) x(v) +WHERE citext_hash(v)::bit(32) != citext_hash_extended(v, 0)::bit(32) + OR citext_hash(v)::bit(32) = citext_hash_extended(v, 1)::bit(32); + value | standard | extended0 | extended1 +-------+----------+-----------+----------- +(0 rows) + -- Do some tests using a table and index. CREATE TEMP TABLE try ( name citext PRIMARY KEY diff --git a/contrib/citext/sql/citext.sql b/contrib/citext/sql/citext.sql index 2732be436d..0cc909eb52 100644 --- a/contrib/citext/sql/citext.sql +++ b/contrib/citext/sql/citext.sql @@ -89,6 +89,15 @@ SELECT citext_cmp('aardvark'::citext, 'aardVark'::citext) AS zero; SELECT citext_cmp('AARDVARK'::citext, 'AARDVARK'::citext) AS zero; SELECT citext_cmp('B'::citext, 'a'::citext) > 0 AS true; +-- Check the citext_hash() and citext_hash_extended() function explicitly. +SELECT v as value, citext_hash(v)::bit(32) as standard, + citext_hash_extended(v, 0)::bit(32) as extended0, + citext_hash_extended(v, 1)::bit(32) as extended1 +FROM (VALUES (NULL::citext), ('PostgreSQL'), ('eIpUEtqmY89'), ('AXKEJBTK'), + ('muop28x03'), ('yi3nm0d73')) x(v) +WHERE citext_hash(v)::bit(32) != citext_hash_extended(v, 0)::bit(32) + OR citext_hash(v)::bit(32) = citext_hash_extended(v, 1)::bit(32); + -- Do some tests using a table and index. CREATE TEMP TABLE try ( -- 2.18.0 ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: 64-bit hash function for hstore and citext data type @ 2018-11-21 05:03 Hironobu SUZUKI <[email protected]> parent: amul sul <[email protected]> 1 sibling, 1 reply; 35+ messages in thread From: Hironobu SUZUKI @ 2018-11-21 05:03 UTC (permalink / raw) To: pgsql-hackers; +Cc: amul sul <[email protected]> On 2018/09/26 11:20, amul sul wrote: > Hi all, > > Commit[1] has added 64-bit hash functions for core data types and in the same > discussion thread[2] Robert Haas suggested to have the similar extended hash > function for hstore and citext data type. Attaching patch proposes the same. > > Regards, > Amul > > 1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=81c5e46c490e2426db243eada186995da... > 2] http://postgr.es/m/CA+Tgmoafx2yoJuhCQQOL5CocEi-w_uG4S2xT0EtgiJnPGcHW3g@mail.gmail.com > I reviewed citext-add-extended-hash-function-v1.patch and hstore-add-extended-hash-function-v1.patch. I could patch and test them without trouble and could not find any issues. I think both patches are well. Best regards, ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: 64-bit hash function for hstore and citext data type @ 2018-11-21 05:09 amul sul <[email protected]> parent: Hironobu SUZUKI <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: amul sul @ 2018-11-21 05:09 UTC (permalink / raw) To: [email protected]; +Cc: pgsql-hackers On Wed, Nov 21, 2018 at 10:34 AM Hironobu SUZUKI <[email protected]> wrote: > > On 2018/09/26 11:20, amul sul wrote: > > Hi all, > > > > Commit[1] has added 64-bit hash functions for core data types and in the same > > discussion thread[2] Robert Haas suggested to have the similar extended hash > > function for hstore and citext data type. Attaching patch proposes the same. > > > > Regards, > > Amul > > > > 1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=81c5e46c490e2426db243eada186995da... > > 2] http://postgr.es/m/CA+Tgmoafx2yoJuhCQQOL5CocEi-w_uG4S2xT0EtgiJnPGcHW3g@mail.gmail.com > > > > > I reviewed citext-add-extended-hash-function-v1.patch and > hstore-add-extended-hash-function-v1.patch. > > I could patch and test them without trouble and could not find any issues. > Thanks to looking at the patch. Regards, Amul ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: 64-bit hash function for hstore and citext data type @ 2018-11-22 18:51 Tomas Vondra <[email protected]> parent: amul sul <[email protected]> 1 sibling, 1 reply; 35+ messages in thread From: Tomas Vondra @ 2018-11-22 18:51 UTC (permalink / raw) To: amul sul <[email protected]>; pgsql-hackers On 9/26/18 12:20 PM, amul sul wrote: > Hi all, > > Commit[1] has added 64-bit hash functions for core data types and in the same > discussion thread[2] Robert Haas suggested to have the similar extended hash > function for hstore and citext data type. Attaching patch proposes the same. > I wonder if the hstore hash function is actually correct. I see it pretty much just computes hash on the varlena representation. The important question is - can there be two different encodings for the same hstore value? If that's possible, those two versions would end up with a different hash value, breaking the hashing scheme. I'm not very familiar with hstore internals so I don't know if that's actually possible, but if you look at hstore_cmp, that seems to be far more complex than just comparing the varlena values directly. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: 64-bit hash function for hstore and citext data type @ 2018-11-22 20:29 Andrew Gierth <[email protected]> parent: Tomas Vondra <[email protected]> 0 siblings, 1 reply; 35+ messages in thread From: Andrew Gierth @ 2018-11-22 20:29 UTC (permalink / raw) To: Tomas Vondra <[email protected]>; +Cc: amul sul <[email protected]>; pgsql-hackers >>>>> "Tomas" == Tomas Vondra <[email protected]> writes: Tomas> I wonder if the hstore hash function is actually correct. I see Tomas> it pretty much just computes hash on the varlena representation. Tomas> The important question is - can there be two different encodings Tomas> for the same hstore value? I was going to say "no", but in fact on closer examination there is an edge case caused by the fact that hstoreUpgrade allows an _empty_ hstore from pg_upgraded 8.4 data through without modifying it. (There's also a vanishingly unlikely case involving the pgfoundry release of hstore-new.) I'm inclined to fix this in hstoreUpgrade rather than complicate hstore_hash with historical trivia. Also there have been no field complaints - I guess it's unlikely that there is much pg 8.4 hstore data in the wild that anyone wants to hash. -- Andrew (irc:RhodiumToad) ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: 64-bit hash function for hstore and citext data type @ 2018-11-23 18:45 Tom Lane <[email protected]> parent: Andrew Gierth <[email protected]> 0 siblings, 2 replies; 35+ messages in thread From: Tom Lane @ 2018-11-23 18:45 UTC (permalink / raw) To: Andrew Gierth <[email protected]>; +Cc: Tomas Vondra <[email protected]>; amul sul <[email protected]>; pgsql-hackers Andrew Gierth <[email protected]> writes: > "Tomas" == Tomas Vondra <[email protected]> writes: > Tomas> The important question is - can there be two different encodings > Tomas> for the same hstore value? > I was going to say "no", but in fact on closer examination there is an > edge case caused by the fact that hstoreUpgrade allows an _empty_ hstore > from pg_upgraded 8.4 data through without modifying it. (There's also a > vanishingly unlikely case involving the pgfoundry release of hstore-new.) Ugh. Still, that's a pre-existing problem in hstore_hash, and so I don't think it's a blocker for this patch. > I'm inclined to fix this in hstoreUpgrade rather than complicate > hstore_hash with historical trivia. Also there have been no field > complaints - I guess it's unlikely that there is much pg 8.4 hstore data > in the wild that anyone wants to hash. Changing hstoreUpgrade at this point seems like wasted/misguided effort. I don't doubt that there was a lot of 8.4 hstore data out there, but how much remains unmigrated? If we're going to take this seriously at all, my inclination would be to change hstore_hash[_extended] to test for the empty-hstore case and force the same value it gets for such an hstore made today. In the meantime, I went ahead and pushed these patches. The only non-cosmetic changes I made were to remove the changes in citext--unpackaged--1.0.sql/hstore--unpackaged--1.0.sql; those were wrong, because the point of those files is to migrate pre-9.1 databases into the extension system. Such a database would not contain an extended hash function, and so adding an ALTER EXTENSION command for that function would cause the script to fail. regards, tom lane ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: 64-bit hash function for hstore and citext data type @ 2018-11-23 19:05 Tom Lane <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 1 reply; 35+ messages in thread From: Tom Lane @ 2018-11-23 19:05 UTC (permalink / raw) To: Andrew Gierth <[email protected]>; +Cc: Tomas Vondra <[email protected]>; amul sul <[email protected]>; pgsql-hackers I wrote: > Andrew Gierth <[email protected]> writes: >> I'm inclined to fix this in hstoreUpgrade rather than complicate >> hstore_hash with historical trivia. Also there have been no field >> complaints - I guess it's unlikely that there is much pg 8.4 hstore data >> in the wild that anyone wants to hash. > Changing hstoreUpgrade at this point seems like wasted/misguided effort. Oh, cancel that --- I was having a momentary brain fade about how that function is used. I agree your proposal is sensible. regards, tom lane ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: 64-bit hash function for hstore and citext data type @ 2018-11-24 11:36 Andrew Gierth <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Andrew Gierth @ 2018-11-24 11:36 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Tomas Vondra <[email protected]>; amul sul <[email protected]>; pgsql-hackers >>>>> "Tom" == Tom Lane <[email protected]> writes: >>> I'm inclined to fix this in hstoreUpgrade rather than complicate >>> hstore_hash with historical trivia. Also there have been no field >>> complaints - I guess it's unlikely that there is much pg 8.4 hstore >>> data in the wild that anyone wants to hash. >> Changing hstoreUpgrade at this point seems like wasted/misguided effort. Tom> Oh, cancel that --- I was having a momentary brain fade about how Tom> that function is used. I agree your proposal is sensible. Here's what I have queued up to push: -- Andrew (irc:RhodiumToad) ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: 64-bit hash function for hstore and citext data type @ 2018-11-26 04:50 amul sul <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 0 replies; 35+ messages in thread From: amul sul @ 2018-11-26 04:50 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: [email protected]; [email protected]; pgsql-hackers Thanks Tom for enhancing & committing these patches. Regards, Amul On Sat, Nov 24, 2018 at 12:15 AM Tom Lane <[email protected]> wrote: > > Andrew Gierth <[email protected]> writes: > > "Tomas" == Tomas Vondra <[email protected]> writes: > > Tomas> The important question is - can there be two different encodings > > Tomas> for the same hstore value? > > > I was going to say "no", but in fact on closer examination there is an > > edge case caused by the fact that hstoreUpgrade allows an _empty_ hstore > > from pg_upgraded 8.4 data through without modifying it. (There's also a > > vanishingly unlikely case involving the pgfoundry release of hstore-new.) > > Ugh. Still, that's a pre-existing problem in hstore_hash, and so I don't > think it's a blocker for this patch. > > > I'm inclined to fix this in hstoreUpgrade rather than complicate > > hstore_hash with historical trivia. Also there have been no field > > complaints - I guess it's unlikely that there is much pg 8.4 hstore data > > in the wild that anyone wants to hash. > > Changing hstoreUpgrade at this point seems like wasted/misguided effort. > I don't doubt that there was a lot of 8.4 hstore data out there, but how > much remains unmigrated? If we're going to take this seriously at all, > my inclination would be to change hstore_hash[_extended] to test for > the empty-hstore case and force the same value it gets for such an > hstore made today. > > In the meantime, I went ahead and pushed these patches. The only > non-cosmetic changes I made were to remove the changes in > citext--unpackaged--1.0.sql/hstore--unpackaged--1.0.sql; those > were wrong, because the point of those files is to migrate pre-9.1 > databases into the extension system. Such a database would not > contain an extended hash function, and so adding an ALTER EXTENSION > command for that function would cause the script to fail. > > regards, tom lane ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH] Unify error messages @ 2019-04-22 19:45 Alvaro Herrera <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Alvaro Herrera @ 2019-04-22 19:45 UTC (permalink / raw) ... for translatability purposes. --- src/backend/storage/ipc/latch.c | 12 +++++++++--- src/backend/storage/ipc/signalfuncs.c | 4 +++- src/backend/utils/adt/formatting.c | 9 ++++++--- src/backend/utils/adt/genfile.c | 4 +++- src/backend/utils/adt/json.c | 4 +++- src/backend/utils/adt/jsonb.c | 4 +++- 6 files changed, 27 insertions(+), 10 deletions(-) diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c index 59fa917ae04..e0712f906a1 100644 --- a/src/backend/storage/ipc/latch.c +++ b/src/backend/storage/ipc/latch.c @@ -856,7 +856,9 @@ WaitEventAdjustEpoll(WaitEventSet *set, WaitEvent *event, int action) if (rc < 0) ereport(ERROR, (errcode_for_socket_access(), - errmsg("epoll_ctl() failed: %m"))); + /* translator: %s is a syscall name, such as "poll()" */ + errmsg("%s failed: %m", + "epoll_ctl()"))); } #endif @@ -1087,7 +1089,9 @@ WaitEventSetWaitBlock(WaitEventSet *set, int cur_timeout, waiting = false; ereport(ERROR, (errcode_for_socket_access(), - errmsg("epoll_wait() failed: %m"))); + /* translator: %s is a syscall name, such as "poll()" */ + errmsg("%s failed: %m", + "epoll_wait()"))); } return 0; } @@ -1211,7 +1215,9 @@ WaitEventSetWaitBlock(WaitEventSet *set, int cur_timeout, waiting = false; ereport(ERROR, (errcode_for_socket_access(), - errmsg("poll() failed: %m"))); + /* translator: %s is a syscall name, such as "poll()" */ + errmsg("%s failed: %m", + "poll()"))); } return 0; } diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c index 4769b1b51eb..4bfbd57464c 100644 --- a/src/backend/storage/ipc/signalfuncs.c +++ b/src/backend/storage/ipc/signalfuncs.c @@ -181,7 +181,9 @@ pg_rotate_logfile(PG_FUNCTION_ARGS) ereport(ERROR, (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), (errmsg("must be superuser to rotate log files with adminpack 1.0"), - errhint("Consider using pg_logfile_rotate(), which is part of core, instead.")))); + /* translator: %s is a SQL function name */ + errhint("Consider using %s, which is part of core, instead.", + "pg_logfile_rotate()")))); if (!Logging_collector) { diff --git a/src/backend/utils/adt/formatting.c b/src/backend/utils/adt/formatting.c index df1db7bc9f1..69a691f18e7 100644 --- a/src/backend/utils/adt/formatting.c +++ b/src/backend/utils/adt/formatting.c @@ -1566,7 +1566,8 @@ str_tolower(const char *buff, size_t nbytes, Oid collid) */ ereport(ERROR, (errcode(ERRCODE_INDETERMINATE_COLLATION), - errmsg("could not determine which collation to use for lower() function"), + errmsg("could not determine which collation to use for %s function", + "lower()"), errhint("Use the COLLATE clause to set the collation explicitly."))); } mylocale = pg_newlocale_from_collation(collid); @@ -1688,7 +1689,8 @@ str_toupper(const char *buff, size_t nbytes, Oid collid) */ ereport(ERROR, (errcode(ERRCODE_INDETERMINATE_COLLATION), - errmsg("could not determine which collation to use for upper() function"), + errmsg("could not determine which collation to use for %s function", + "upper()"), errhint("Use the COLLATE clause to set the collation explicitly."))); } mylocale = pg_newlocale_from_collation(collid); @@ -1811,7 +1813,8 @@ str_initcap(const char *buff, size_t nbytes, Oid collid) */ ereport(ERROR, (errcode(ERRCODE_INDETERMINATE_COLLATION), - errmsg("could not determine which collation to use for initcap() function"), + errmsg("could not determine which collation to use for %s function", + "initcap()"), errhint("Use the COLLATE clause to set the collation explicitly."))); } mylocale = pg_newlocale_from_collation(collid); diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index d6976609968..a3c6adaf640 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -219,7 +219,9 @@ pg_read_file(PG_FUNCTION_ARGS) ereport(ERROR, (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), (errmsg("must be superuser to read files with adminpack 1.0"), - errhint("Consider using pg_file_read(), which is part of core, instead.")))); + /* translator: %s is a SQL function name */ + errhint("Consider using %s, which is part of core, instead.", + "pg_file_read()")))); /* handle optional arguments */ if (PG_NARGS() >= 3) diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c index de0d0723b71..bb4bac85f7d 100644 --- a/src/backend/utils/adt/json.c +++ b/src/backend/utils/adt/json.c @@ -2192,7 +2192,9 @@ json_build_object(PG_FUNCTION_ARGS) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("argument list must have even number of elements"), - errhint("The arguments of json_build_object() must consist of alternating keys and values."))); + /* translator: %s is a SQL function name */ + errhint("The arguments of %s must consist of alternating keys and values.", + "json_build_object()"))); result = makeStringInfo(); diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c index 7af4091200b..036d771386f 100644 --- a/src/backend/utils/adt/jsonb.c +++ b/src/backend/utils/adt/jsonb.c @@ -1155,7 +1155,9 @@ jsonb_build_object(PG_FUNCTION_ARGS) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("argument list must have even number of elements"), - errhint("The arguments of jsonb_build_object() must consist of alternating keys and values."))); + /* translator: %s is a SQL function name */ + errhint("The arguments of %s must consist of alternating keys and values.", + "jsonb_build_object()"))); memset(&result, 0, sizeof(JsonbInState)); -- 2.17.1 --W/nzBZO5zC0uMSeA-- ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH] Change pg_restore -f- to dump to stdout instead of to ./- @ 2019-11-04 18:50 Alvaro Herrera <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Alvaro Herrera @ 2019-11-04 18:50 UTC (permalink / raw) Starting with PostgreSQL 12, pg_restore refuses to run when neither -d nor -f are specified (c.f. commit 413ccaa74d9a), and it also makes "-f -" mean the old implicit behavior of dumping to stdout. However, older branches write to a file called ./- when invoked like that, making it impossible to write pg_restore scripts that work across versions. This is a partial backpatch of the aforementioned commit to all older supported branches, providing an upgrade path. Discussion: https://postgr.es/m/[email protected] --- doc/src/sgml/ref/pg_restore.sgml | 4 ++-- src/bin/pg_dump/pg_backup_archiver.c | 7 ++++++- src/bin/pg_dump/pg_restore.c | 2 +- 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml index 725acb192c..18c9257ae9 100644 --- a/doc/src/sgml/ref/pg_restore.sgml +++ b/doc/src/sgml/ref/pg_restore.sgml @@ -176,8 +176,8 @@ <listitem> <para> Specify output file for generated script, or for the listing - when used with <option>-l</option>. Default is the standard - output. + when used with <option>-l</option>. Use <literal>-</literal> + for <systemitem>stdout</systemitem>. </para> </listitem> </varlistentry> diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c index 01b4af64f6..1ebbe852bc 100644 --- a/src/bin/pg_dump/pg_backup_archiver.c +++ b/src/bin/pg_dump/pg_backup_archiver.c @@ -1511,7 +1511,12 @@ SetOutput(ArchiveHandle *AH, const char *filename, int compression) int fn; if (filename) - fn = -1; + { + if (strcmp(filename, "-") == 0) + fn = fileno(stdout); + else + fn = -1; + } else if (AH->FH) fn = fileno(AH->FH); else if (AH->fSpec) diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c index 34d93ab472..f5df4e63d7 100644 --- a/src/bin/pg_dump/pg_restore.c +++ b/src/bin/pg_dump/pg_restore.c @@ -454,7 +454,7 @@ usage(const char *progname) printf(_("\nGeneral options:\n")); printf(_(" -d, --dbname=NAME connect to database name\n")); - printf(_(" -f, --file=FILENAME output file name\n")); + printf(_(" -f, --file=FILENAME output file name (- for stdout)\n")); printf(_(" -F, --format=c|d|t backup file format (should be automatic)\n")); printf(_(" -l, --list print summarized TOC of the archive\n")); printf(_(" -v, --verbose verbose mode\n")); -- 2.20.1 --EVF5PPMfhYS0aIcm-- ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v3 1/2] partial revert of ff9618e82a @ 2023-06-14 17:54 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-14 17:54 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 11 +------ doc/src/sgml/ref/cluster.sgml | 9 +----- doc/src/sgml/ref/lock.sgml | 11 ++----- .../sgml/ref/refresh_materialized_view.sgml | 6 ++-- doc/src/sgml/ref/reindex.sgml | 27 +++++++---------- doc/src/sgml/ref/vacuum.sgml | 11 +------ doc/src/sgml/user-manag.sgml | 3 +- src/backend/commands/cluster.c | 10 ++----- src/backend/commands/indexcmds.c | 17 ++++------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 29 +------------------ src/backend/commands/vacuum.c | 6 +--- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++----- .../specs/cluster-conflict-partition.spec | 7 +++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/vacuum.out | 18 ++++++++++++ 17 files changed, 61 insertions(+), 130 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..954491b5df 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -183,17 +183,8 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea <para> To analyze a table, one must ordinarily have the <literal>MAINTAIN</literal> - privilege on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. However, database owners are allowed to + privilege on the table. However, database owners are allowed to analyze all tables in their databases, except shared catalogs. - (The restriction for shared catalogs means that a true database-wide - <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..06f3d269e6 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -134,14 +134,7 @@ CLUSTER [VERBOSE] <para> To cluster a table, one must have the <literal>MAINTAIN</literal> privilege - on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any - tables that the calling user does not have permission to cluster. + on the table. </para> <para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..070855da18 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -166,10 +166,8 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] <para> To lock a table, the user must have the right privilege for the specified - <replaceable class="parameter">lockmode</replaceable>, or be the table's - owner, a superuser, or a role with privileges of the <link - linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If the user has <literal>MAINTAIN</literal>, + <replaceable class="parameter">lockmode</replaceable>. + If the user has <literal>MAINTAIN</literal>, <literal>UPDATE</literal>, <literal>DELETE</literal>, or <literal>TRUNCATE</literal> privileges on the table, any <replaceable class="parameter">lockmode</replaceable> is permitted. If the user has @@ -177,10 +175,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/refresh_materialized_view.sgml b/doc/src/sgml/ref/refresh_materialized_view.sgml index 4d79b6ae7f..19737668cd 100644 --- a/doc/src/sgml/ref/refresh_materialized_view.sgml +++ b/doc/src/sgml/ref/refresh_materialized_view.sgml @@ -31,10 +31,8 @@ REFRESH MATERIALIZED VIEW [ CONCURRENTLY ] <replaceable class="parameter">name</ <para> <command>REFRESH MATERIALIZED VIEW</command> completely replaces the - contents of a materialized view. To execute this command you must be the - owner of the materialized view, have privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role, or have the <literal>MAINTAIN</literal> + contents of a materialized view. To execute this command you must have the + <literal>MAINTAIN</literal> privilege on the materialized view. The old contents are discarded. If <literal>WITH DATA</literal> is specified (or defaults) the backing query is executed to provide the new data, and the materialized view is left in a diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..583553b8a3 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -292,25 +292,18 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA </para> <para> - Reindexing a single index or table requires being the owner of that - index or table, having privileges of the + Reindexing a single index or table requires having the + <literal>MAINTAIN</literal> privilege on the table. Reindexing a schema or + database requires being the owner of that schema or database or having + privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role, or having the <literal>MAINTAIN</literal> privilege on the - table. Reindexing a schema or database requires being the - owner of that schema or database or having privileges of the - <literal>pg_maintain</literal> role. Note specifically that it's thus + role. Note specifically that it's thus possible for non-superusers to rebuild indexes of tables owned by - other users. However, as a special exception, when - <command>REINDEX DATABASE</command>, <command>REINDEX SCHEMA</command> - or <command>REINDEX SYSTEM</command> is issued by a non-superuser, - indexes on shared catalogs will be skipped unless the user owns the - catalog (which typically won't be the case), has privileges of the - <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + other users. However, as a special exception, + <command>REINDEX DATABASE</command>, <command>REINDEX SCHEMA</command>, and + <command>REINDEX SYSTEM</command> will skip indexes on shared catalogs + unless the user has the <literal>MAINTAIN</literal> privilege on the + catalog. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..c42bbea9e2 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -445,17 +445,8 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet <para> To vacuum a table, one must ordinarily have the <literal>MAINTAIN</literal> - privilege on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. However, database owners are allowed to + privilege on the table. However, database owners are allowed to vacuum all tables in their databases, except shared catalogs. - (The restriction for shared catalogs means that a true database-wide - <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/doc/src/sgml/user-manag.sgml b/doc/src/sgml/user-manag.sgml index b6c37ccef2..e1540dd481 100644 --- a/doc/src/sgml/user-manag.sgml +++ b/doc/src/sgml/user-manag.sgml @@ -692,7 +692,8 @@ DROP ROLE doomed_role; <link linkend="sql-refreshmaterializedview"><command>REFRESH MATERIALIZED VIEW</command></link>, <link linkend="sql-reindex"><command>REINDEX</command></link>, and <link linkend="sql-lock"><command>LOCK TABLE</command></link> on all - relations.</entry> + relations, as if having <literal>MAINTAIN</literal> rights on those + objects, even without having it explicitly.</entry> </row> <row> <entry>pg_use_reserved_connections</entry> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..ed28d13a16 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2854,8 +2854,7 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, relation->relname); @@ -3064,18 +3063,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..29eece1c2c 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -17006,38 +17006,11 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) + if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, relation->relname); } -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; -} - /* * Callback to RangeVarGetRelidExtended() for TRUNCATE processing. */ diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index a843f9ad92..2e41a31173 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -719,13 +718,10 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --sdtB3X0nJg68CQEu Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0002-convert-boolean-parameter-into-a-VACOPT_-option.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v1 1/2] partial revert of ff9618e82a @ 2023-06-14 17:54 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-14 17:54 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +--- doc/src/sgml/ref/cluster.sgml | 6 +--- doc/src/sgml/ref/lock.sgml | 5 +--- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +--- src/backend/commands/cluster.c | 10 ++----- src/backend/commands/indexcmds.c | 19 +++++------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 29 +------------------ src/backend/commands/vacuum.c | 6 +--- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++----- .../specs/cluster-conflict-partition.spec | 7 +++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/vacuum.out | 18 ++++++++++++ 15 files changed, 49 insertions(+), 93 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..c768175499 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,11 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any - tables that the calling user does not have permission to cluster. + role. </para> <para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..56bba91c08 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2854,8 +2854,7 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, relation->relname); @@ -3065,17 +3064,15 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, /* * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * the table or the user is a superuser, the table owner, or the + * database/schema owner (but in the latter case, only if it's not a + * shared relation). pg_class_aclcheck includes the superuser case, + * and depending on objectKind we already know that the user has + * permission to run REINDEX on this database or schema per the + * permission checks at the beginning of this routine. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..29eece1c2c 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -17006,38 +17006,11 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) + if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, relation->relname); } -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; -} - /* * Callback to RangeVarGetRelidExtended() for TRUNCATE processing. */ diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index a843f9ad92..2e41a31173 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -719,13 +718,10 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --a8Wt8u1KmwUX3Y2C Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v1-0002-convert-boolean-parameter-into-a-VACOPT_-option.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v2 1/2] partial revert of ff9618e82a @ 2023-06-14 17:54 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-14 17:54 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 13 ++------- doc/src/sgml/ref/cluster.sgml | 9 +----- doc/src/sgml/ref/lock.sgml | 24 ++++++--------- .../sgml/ref/refresh_materialized_view.sgml | 17 +++++------ doc/src/sgml/ref/reindex.sgml | 28 +++++++----------- doc/src/sgml/ref/vacuum.sgml | 13 ++------- doc/src/sgml/user-manag.sgml | 3 +- src/backend/commands/cluster.c | 10 ++----- src/backend/commands/indexcmds.c | 17 ++++------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 29 +------------------ src/backend/commands/vacuum.c | 6 +--- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++----- .../specs/cluster-conflict-partition.spec | 7 +++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/vacuum.out | 18 ++++++++++++ 17 files changed, 74 insertions(+), 146 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..ecc7c884b4 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -183,17 +183,8 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea <para> To analyze a table, one must ordinarily have the <literal>MAINTAIN</literal> - privilege on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. However, database owners are allowed to - analyze all tables in their databases, except shared catalogs. - (The restriction for shared catalogs means that a true database-wide - <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + privilege on the table. However, database owners are allowed to analyze all + tables in their databases, except shared catalogs. <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..06f3d269e6 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -134,14 +134,7 @@ CLUSTER [VERBOSE] <para> To cluster a table, one must have the <literal>MAINTAIN</literal> privilege - on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any - tables that the calling user does not have permission to cluster. + on the table. </para> <para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..d22c6f8384 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -166,21 +166,15 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] <para> To lock a table, the user must have the right privilege for the specified - <replaceable class="parameter">lockmode</replaceable>, or be the table's - owner, a superuser, or a role with privileges of the <link - linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If the user has <literal>MAINTAIN</literal>, - <literal>UPDATE</literal>, <literal>DELETE</literal>, or - <literal>TRUNCATE</literal> privileges on the table, any <replaceable - class="parameter">lockmode</replaceable> is permitted. If the user has - <literal>INSERT</literal> privileges on the table, <literal>ROW EXCLUSIVE - MODE</literal> (or a less-conflicting mode as described in <xref - linkend="explicit-locking"/>) is permitted. If a user has - <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + <replaceable class="parameter">lockmode</replaceable>. If the user has + <literal>MAINTAIN</literal>, <literal>UPDATE</literal>, + <literal>DELETE</literal>, or <literal>TRUNCATE</literal> privileges on the + table, any <replaceable class="parameter">lockmode</replaceable> is + permitted. If the user has <literal>INSERT</literal> privileges on the + table, <literal>ROW EXCLUSIVE MODE</literal> (or a less-conflicting mode as + described in <xref linkend="explicit-locking"/>) is permitted. If a user + has <literal>SELECT</literal> privileges on the table, + <literal>ACCESS SHARE MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/refresh_materialized_view.sgml b/doc/src/sgml/ref/refresh_materialized_view.sgml index 4d79b6ae7f..199a577d36 100644 --- a/doc/src/sgml/ref/refresh_materialized_view.sgml +++ b/doc/src/sgml/ref/refresh_materialized_view.sgml @@ -31,16 +31,13 @@ REFRESH MATERIALIZED VIEW [ CONCURRENTLY ] <replaceable class="parameter">name</ <para> <command>REFRESH MATERIALIZED VIEW</command> completely replaces the - contents of a materialized view. To execute this command you must be the - owner of the materialized view, have privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role, or have the <literal>MAINTAIN</literal> - privilege on the materialized view. The old contents are discarded. If - <literal>WITH DATA</literal> is specified (or defaults) the backing query - is executed to provide the new data, and the materialized view is left in a - scannable state. If <literal>WITH NO DATA</literal> is specified no new - data is generated and the materialized view is left in an unscannable - state. + contents of a materialized view. To execute this command you must have the + <literal>MAINTAIN</literal> privilege on the materialized view. The old + contents are discarded. If <literal>WITH DATA</literal> is specified (or + defaults) the backing query is executed to provide the new data, and the + materialized view is left in a scannable state. If + <literal>WITH NO DATA</literal> is specified no new data is generated and + the materialized view is left in an unscannable state. </para> <para> <literal>CONCURRENTLY</literal> and <literal>WITH NO DATA</literal> may not diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..a64b021e66 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -292,25 +292,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA </para> <para> - Reindexing a single index or table requires being the owner of that - index or table, having privileges of the + Reindexing a single index or table requires having the + <literal>MAINTAIN</literal> privilege on the table. Reindexing a schema or + database requires being the owner of that schema or database or having + privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role, or having the <literal>MAINTAIN</literal> privilege on the - table. Reindexing a schema or database requires being the - owner of that schema or database or having privileges of the - <literal>pg_maintain</literal> role. Note specifically that it's thus - possible for non-superusers to rebuild indexes of tables owned by - other users. However, as a special exception, when - <command>REINDEX DATABASE</command>, <command>REINDEX SCHEMA</command> - or <command>REINDEX SYSTEM</command> is issued by a non-superuser, - indexes on shared catalogs will be skipped unless the user owns the - catalog (which typically won't be the case), has privileges of the - <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + role. Note specifically that it's thus possible for non-superusers to + rebuild indexes of tables owned by other users. However, as a special + exception, <command>REINDEX DATABASE</command>, + <command>REINDEX SCHEMA</command>, and <command>REINDEX SYSTEM</command> + will skip indexes on shared catalogs unless the user has the + <literal>MAINTAIN</literal> privilege on the catalog. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..fa2ee76e25 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -445,17 +445,8 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet <para> To vacuum a table, one must ordinarily have the <literal>MAINTAIN</literal> - privilege on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. However, database owners are allowed to - vacuum all tables in their databases, except shared catalogs. - (The restriction for shared catalogs means that a true database-wide - <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + privilege on the table. However, database owners are allowed to vacuum all + tables in their databases, except shared catalogs. <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/doc/src/sgml/user-manag.sgml b/doc/src/sgml/user-manag.sgml index b6c37ccef2..e1540dd481 100644 --- a/doc/src/sgml/user-manag.sgml +++ b/doc/src/sgml/user-manag.sgml @@ -692,7 +692,8 @@ DROP ROLE doomed_role; <link linkend="sql-refreshmaterializedview"><command>REFRESH MATERIALIZED VIEW</command></link>, <link linkend="sql-reindex"><command>REINDEX</command></link>, and <link linkend="sql-lock"><command>LOCK TABLE</command></link> on all - relations.</entry> + relations, as if having <literal>MAINTAIN</literal> rights on those + objects, even without having it explicitly.</entry> </row> <row> <entry>pg_use_reserved_connections</entry> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..ed28d13a16 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2854,8 +2854,7 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, relation->relname); @@ -3064,18 +3063,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..29eece1c2c 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -17006,38 +17006,11 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) + if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, relation->relname); } -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; -} - /* * Callback to RangeVarGetRelidExtended() for TRUNCATE processing. */ diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index a843f9ad92..2e41a31173 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -719,13 +718,10 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --EVF5PPMfhYS0aIcm Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0002-convert-boolean-parameter-into-a-VACOPT_-option.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v3 1/2] partial revert of ff9618e82a @ 2023-06-14 17:54 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-14 17:54 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 11 +------ doc/src/sgml/ref/cluster.sgml | 9 +----- doc/src/sgml/ref/lock.sgml | 11 ++----- .../sgml/ref/refresh_materialized_view.sgml | 6 ++-- doc/src/sgml/ref/reindex.sgml | 27 +++++++---------- doc/src/sgml/ref/vacuum.sgml | 11 +------ doc/src/sgml/user-manag.sgml | 3 +- src/backend/commands/cluster.c | 10 ++----- src/backend/commands/indexcmds.c | 17 ++++------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 29 +------------------ src/backend/commands/vacuum.c | 6 +--- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++----- .../specs/cluster-conflict-partition.spec | 7 +++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/vacuum.out | 18 ++++++++++++ 17 files changed, 61 insertions(+), 130 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..954491b5df 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -183,17 +183,8 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea <para> To analyze a table, one must ordinarily have the <literal>MAINTAIN</literal> - privilege on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. However, database owners are allowed to + privilege on the table. However, database owners are allowed to analyze all tables in their databases, except shared catalogs. - (The restriction for shared catalogs means that a true database-wide - <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..06f3d269e6 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -134,14 +134,7 @@ CLUSTER [VERBOSE] <para> To cluster a table, one must have the <literal>MAINTAIN</literal> privilege - on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any - tables that the calling user does not have permission to cluster. + on the table. </para> <para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..070855da18 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -166,10 +166,8 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] <para> To lock a table, the user must have the right privilege for the specified - <replaceable class="parameter">lockmode</replaceable>, or be the table's - owner, a superuser, or a role with privileges of the <link - linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If the user has <literal>MAINTAIN</literal>, + <replaceable class="parameter">lockmode</replaceable>. + If the user has <literal>MAINTAIN</literal>, <literal>UPDATE</literal>, <literal>DELETE</literal>, or <literal>TRUNCATE</literal> privileges on the table, any <replaceable class="parameter">lockmode</replaceable> is permitted. If the user has @@ -177,10 +175,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/refresh_materialized_view.sgml b/doc/src/sgml/ref/refresh_materialized_view.sgml index 4d79b6ae7f..19737668cd 100644 --- a/doc/src/sgml/ref/refresh_materialized_view.sgml +++ b/doc/src/sgml/ref/refresh_materialized_view.sgml @@ -31,10 +31,8 @@ REFRESH MATERIALIZED VIEW [ CONCURRENTLY ] <replaceable class="parameter">name</ <para> <command>REFRESH MATERIALIZED VIEW</command> completely replaces the - contents of a materialized view. To execute this command you must be the - owner of the materialized view, have privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role, or have the <literal>MAINTAIN</literal> + contents of a materialized view. To execute this command you must have the + <literal>MAINTAIN</literal> privilege on the materialized view. The old contents are discarded. If <literal>WITH DATA</literal> is specified (or defaults) the backing query is executed to provide the new data, and the materialized view is left in a diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..583553b8a3 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -292,25 +292,18 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA </para> <para> - Reindexing a single index or table requires being the owner of that - index or table, having privileges of the + Reindexing a single index or table requires having the + <literal>MAINTAIN</literal> privilege on the table. Reindexing a schema or + database requires being the owner of that schema or database or having + privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role, or having the <literal>MAINTAIN</literal> privilege on the - table. Reindexing a schema or database requires being the - owner of that schema or database or having privileges of the - <literal>pg_maintain</literal> role. Note specifically that it's thus + role. Note specifically that it's thus possible for non-superusers to rebuild indexes of tables owned by - other users. However, as a special exception, when - <command>REINDEX DATABASE</command>, <command>REINDEX SCHEMA</command> - or <command>REINDEX SYSTEM</command> is issued by a non-superuser, - indexes on shared catalogs will be skipped unless the user owns the - catalog (which typically won't be the case), has privileges of the - <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + other users. However, as a special exception, + <command>REINDEX DATABASE</command>, <command>REINDEX SCHEMA</command>, and + <command>REINDEX SYSTEM</command> will skip indexes on shared catalogs + unless the user has the <literal>MAINTAIN</literal> privilege on the + catalog. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..c42bbea9e2 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -445,17 +445,8 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet <para> To vacuum a table, one must ordinarily have the <literal>MAINTAIN</literal> - privilege on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. However, database owners are allowed to + privilege on the table. However, database owners are allowed to vacuum all tables in their databases, except shared catalogs. - (The restriction for shared catalogs means that a true database-wide - <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/doc/src/sgml/user-manag.sgml b/doc/src/sgml/user-manag.sgml index b6c37ccef2..e1540dd481 100644 --- a/doc/src/sgml/user-manag.sgml +++ b/doc/src/sgml/user-manag.sgml @@ -692,7 +692,8 @@ DROP ROLE doomed_role; <link linkend="sql-refreshmaterializedview"><command>REFRESH MATERIALIZED VIEW</command></link>, <link linkend="sql-reindex"><command>REINDEX</command></link>, and <link linkend="sql-lock"><command>LOCK TABLE</command></link> on all - relations.</entry> + relations, as if having <literal>MAINTAIN</literal> rights on those + objects, even without having it explicitly.</entry> </row> <row> <entry>pg_use_reserved_connections</entry> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..ed28d13a16 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2854,8 +2854,7 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, relation->relname); @@ -3064,18 +3063,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..29eece1c2c 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -17006,38 +17006,11 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) + if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, relation->relname); } -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; -} - /* * Callback to RangeVarGetRelidExtended() for TRUNCATE processing. */ diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index a843f9ad92..2e41a31173 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -719,13 +718,10 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --sdtB3X0nJg68CQEu Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0002-convert-boolean-parameter-into-a-VACOPT_-option.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v2 1/2] partial revert of ff9618e82a @ 2023-06-14 17:54 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-14 17:54 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 13 ++------- doc/src/sgml/ref/cluster.sgml | 9 +----- doc/src/sgml/ref/lock.sgml | 24 ++++++--------- .../sgml/ref/refresh_materialized_view.sgml | 17 +++++------ doc/src/sgml/ref/reindex.sgml | 28 +++++++----------- doc/src/sgml/ref/vacuum.sgml | 13 ++------- doc/src/sgml/user-manag.sgml | 3 +- src/backend/commands/cluster.c | 10 ++----- src/backend/commands/indexcmds.c | 17 ++++------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 29 +------------------ src/backend/commands/vacuum.c | 6 +--- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++----- .../specs/cluster-conflict-partition.spec | 7 +++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/vacuum.out | 18 ++++++++++++ 17 files changed, 74 insertions(+), 146 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..ecc7c884b4 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -183,17 +183,8 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea <para> To analyze a table, one must ordinarily have the <literal>MAINTAIN</literal> - privilege on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. However, database owners are allowed to - analyze all tables in their databases, except shared catalogs. - (The restriction for shared catalogs means that a true database-wide - <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + privilege on the table. However, database owners are allowed to analyze all + tables in their databases, except shared catalogs. <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..06f3d269e6 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -134,14 +134,7 @@ CLUSTER [VERBOSE] <para> To cluster a table, one must have the <literal>MAINTAIN</literal> privilege - on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any - tables that the calling user does not have permission to cluster. + on the table. </para> <para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..d22c6f8384 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -166,21 +166,15 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] <para> To lock a table, the user must have the right privilege for the specified - <replaceable class="parameter">lockmode</replaceable>, or be the table's - owner, a superuser, or a role with privileges of the <link - linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If the user has <literal>MAINTAIN</literal>, - <literal>UPDATE</literal>, <literal>DELETE</literal>, or - <literal>TRUNCATE</literal> privileges on the table, any <replaceable - class="parameter">lockmode</replaceable> is permitted. If the user has - <literal>INSERT</literal> privileges on the table, <literal>ROW EXCLUSIVE - MODE</literal> (or a less-conflicting mode as described in <xref - linkend="explicit-locking"/>) is permitted. If a user has - <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + <replaceable class="parameter">lockmode</replaceable>. If the user has + <literal>MAINTAIN</literal>, <literal>UPDATE</literal>, + <literal>DELETE</literal>, or <literal>TRUNCATE</literal> privileges on the + table, any <replaceable class="parameter">lockmode</replaceable> is + permitted. If the user has <literal>INSERT</literal> privileges on the + table, <literal>ROW EXCLUSIVE MODE</literal> (or a less-conflicting mode as + described in <xref linkend="explicit-locking"/>) is permitted. If a user + has <literal>SELECT</literal> privileges on the table, + <literal>ACCESS SHARE MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/refresh_materialized_view.sgml b/doc/src/sgml/ref/refresh_materialized_view.sgml index 4d79b6ae7f..199a577d36 100644 --- a/doc/src/sgml/ref/refresh_materialized_view.sgml +++ b/doc/src/sgml/ref/refresh_materialized_view.sgml @@ -31,16 +31,13 @@ REFRESH MATERIALIZED VIEW [ CONCURRENTLY ] <replaceable class="parameter">name</ <para> <command>REFRESH MATERIALIZED VIEW</command> completely replaces the - contents of a materialized view. To execute this command you must be the - owner of the materialized view, have privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role, or have the <literal>MAINTAIN</literal> - privilege on the materialized view. The old contents are discarded. If - <literal>WITH DATA</literal> is specified (or defaults) the backing query - is executed to provide the new data, and the materialized view is left in a - scannable state. If <literal>WITH NO DATA</literal> is specified no new - data is generated and the materialized view is left in an unscannable - state. + contents of a materialized view. To execute this command you must have the + <literal>MAINTAIN</literal> privilege on the materialized view. The old + contents are discarded. If <literal>WITH DATA</literal> is specified (or + defaults) the backing query is executed to provide the new data, and the + materialized view is left in a scannable state. If + <literal>WITH NO DATA</literal> is specified no new data is generated and + the materialized view is left in an unscannable state. </para> <para> <literal>CONCURRENTLY</literal> and <literal>WITH NO DATA</literal> may not diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..a64b021e66 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -292,25 +292,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA </para> <para> - Reindexing a single index or table requires being the owner of that - index or table, having privileges of the + Reindexing a single index or table requires having the + <literal>MAINTAIN</literal> privilege on the table. Reindexing a schema or + database requires being the owner of that schema or database or having + privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role, or having the <literal>MAINTAIN</literal> privilege on the - table. Reindexing a schema or database requires being the - owner of that schema or database or having privileges of the - <literal>pg_maintain</literal> role. Note specifically that it's thus - possible for non-superusers to rebuild indexes of tables owned by - other users. However, as a special exception, when - <command>REINDEX DATABASE</command>, <command>REINDEX SCHEMA</command> - or <command>REINDEX SYSTEM</command> is issued by a non-superuser, - indexes on shared catalogs will be skipped unless the user owns the - catalog (which typically won't be the case), has privileges of the - <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + role. Note specifically that it's thus possible for non-superusers to + rebuild indexes of tables owned by other users. However, as a special + exception, <command>REINDEX DATABASE</command>, + <command>REINDEX SCHEMA</command>, and <command>REINDEX SYSTEM</command> + will skip indexes on shared catalogs unless the user has the + <literal>MAINTAIN</literal> privilege on the catalog. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..fa2ee76e25 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -445,17 +445,8 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet <para> To vacuum a table, one must ordinarily have the <literal>MAINTAIN</literal> - privilege on the table or be the table's owner, a superuser, or a role with - privileges of the - <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. However, database owners are allowed to - vacuum all tables in their databases, except shared catalogs. - (The restriction for shared catalogs means that a true database-wide - <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + privilege on the table. However, database owners are allowed to vacuum all + tables in their databases, except shared catalogs. <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/doc/src/sgml/user-manag.sgml b/doc/src/sgml/user-manag.sgml index b6c37ccef2..e1540dd481 100644 --- a/doc/src/sgml/user-manag.sgml +++ b/doc/src/sgml/user-manag.sgml @@ -692,7 +692,8 @@ DROP ROLE doomed_role; <link linkend="sql-refreshmaterializedview"><command>REFRESH MATERIALIZED VIEW</command></link>, <link linkend="sql-reindex"><command>REINDEX</command></link>, and <link linkend="sql-lock"><command>LOCK TABLE</command></link> on all - relations.</entry> + relations, as if having <literal>MAINTAIN</literal> rights on those + objects, even without having it explicitly.</entry> </row> <row> <entry>pg_use_reserved_connections</entry> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..ed28d13a16 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2854,8 +2854,7 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, relation->relname); @@ -3064,18 +3063,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..29eece1c2c 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -17006,38 +17006,11 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) + if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, relation->relname); } -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; -} - /* * Callback to RangeVarGetRelidExtended() for TRUNCATE processing. */ diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index a843f9ad92..2e41a31173 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -719,13 +718,10 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --EVF5PPMfhYS0aIcm Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0002-convert-boolean-parameter-into-a-VACOPT_-option.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v1 1/2] partial revert of ff9618e82a @ 2023-06-14 17:54 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-14 17:54 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +--- doc/src/sgml/ref/cluster.sgml | 6 +--- doc/src/sgml/ref/lock.sgml | 5 +--- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +--- src/backend/commands/cluster.c | 10 ++----- src/backend/commands/indexcmds.c | 19 +++++------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 29 +------------------ src/backend/commands/vacuum.c | 6 +--- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++----- .../specs/cluster-conflict-partition.spec | 7 +++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/vacuum.out | 18 ++++++++++++ 15 files changed, 49 insertions(+), 93 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..c768175499 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,11 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any - tables that the calling user does not have permission to cluster. + role. </para> <para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..56bba91c08 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2854,8 +2854,7 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, relation->relname); @@ -3065,17 +3064,15 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, /* * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * the table or the user is a superuser, the table owner, or the + * database/schema owner (but in the latter case, only if it's not a + * shared relation). pg_class_aclcheck includes the superuser case, + * and depending on objectKind we already know that the user has + * permission to run REINDEX on this database or schema per the + * permission checks at the beginning of this routine. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..29eece1c2c 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -17006,38 +17006,11 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) + if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, relation->relname); } -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; -} - /* * Callback to RangeVarGetRelidExtended() for TRUNCATE processing. */ diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index a843f9ad92..2e41a31173 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -719,13 +718,10 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --a8Wt8u1KmwUX3Y2C Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v1-0002-convert-boolean-parameter-into-a-VACOPT_-option.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v7 1/2] partial revert of ff9618e82a @ 2023-06-19 20:57 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-19 20:57 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +-- doc/src/sgml/ref/cluster.sgml | 5 +-- doc/src/sgml/ref/lock.sgml | 5 +-- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +-- src/backend/commands/cluster.c | 10 ++---- src/backend/commands/indexcmds.c | 27 +++++++-------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 34 +++---------------- src/backend/commands/vacuum.c | 8 +---- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++---- .../specs/cluster-conflict-partition.spec | 7 ++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/create_index.out | 4 +-- src/test/regress/expected/privileges.out | 8 ++--- src/test/regress/expected/vacuum.out | 18 ++++++++++ 17 files changed, 62 insertions(+), 106 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..f0dd7faed5 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,10 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any + role. <command>CLUSTER</command> will skip over any tables that the calling user does not have permission to cluster. </para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..9bc97e1fc2 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2853,11 +2853,14 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); - if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, - relation->relname); + if (OidIsValid(table_oid)) + { + AclResult aclresult; + + aclresult = pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_INDEX, relation->relname); + } /* Lock heap before index to avoid deadlock. */ if (relId != oldRelId) @@ -3064,18 +3067,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..9b12bc44d7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -16986,6 +16986,7 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg) { char relkind; + AclResult aclresult; /* Nothing to do if the relation was not found. */ if (!OidIsValid(relId)) @@ -17006,36 +17007,9 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, - relation->relname); -} - -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; + aclresult = pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_TABLE, relation->relname); } /* diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index bb79de4da6..e7a0450cc4 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -721,17 +720,12 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, /*---------- * A role has privileges to vacuum or analyze the relation if any of the * following are true: - * - the role is a superuser - * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out index acfd9d1f4f..1473bc3175 100644 --- a/src/test/regress/expected/create_index.out +++ b/src/test/regress/expected/create_index.out @@ -2831,9 +2831,9 @@ RESET ROLE; GRANT USAGE ON SCHEMA pg_toast TO regress_reindexuser; SET SESSION ROLE regress_reindexuser; REINDEX TABLE pg_toast.pg_toast_1260; -ERROR: must be owner of table pg_toast_1260 +ERROR: permission denied for table pg_toast_1260 REINDEX INDEX pg_toast.pg_toast_1260_index; -ERROR: must be owner of index pg_toast_1260_index +ERROR: permission denied for index pg_toast_1260_index -- Clean up RESET ROLE; REVOKE USAGE ON SCHEMA pg_toast FROM regress_reindexuser; diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out index 3cf4ac8c9e..3e4dfcc2ec 100644 --- a/src/test/regress/expected/privileges.out +++ b/src/test/regress/expected/privileges.out @@ -2928,13 +2928,13 @@ WARNING: permission denied to analyze "maintain_test", skipping it VACUUM (ANALYZE) maintain_test; WARNING: permission denied to vacuum "maintain_test", skipping it CLUSTER maintain_test USING maintain_test_a_idx; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REFRESH MATERIALIZED VIEW refresh_test; -ERROR: must be owner of table refresh_test +ERROR: permission denied for table refresh_test REINDEX TABLE maintain_test; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REINDEX INDEX maintain_test_a_idx; -ERROR: must be owner of index maintain_test_a_idx +ERROR: permission denied for index maintain_test_a_idx REINDEX SCHEMA reindex_test; ERROR: must be owner of schema reindex_test RESET ROLE; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --/9DWx/yDrRhgMJTb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0002-simplify-privilege-related-documentation-for-main.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v4 2/3] partial revert of ff9618e82a @ 2023-06-19 20:57 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-19 20:57 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +-- doc/src/sgml/ref/cluster.sgml | 5 +-- doc/src/sgml/ref/lock.sgml | 5 +-- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +-- src/backend/commands/cluster.c | 10 ++---- src/backend/commands/indexcmds.c | 27 +++++++-------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 34 +++---------------- src/backend/commands/vacuum.c | 8 +---- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++---- .../specs/cluster-conflict-partition.spec | 7 ++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/create_index.out | 4 +-- src/test/regress/expected/privileges.out | 8 ++--- src/test/regress/expected/vacuum.out | 18 ++++++++++ 17 files changed, 62 insertions(+), 106 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..f0dd7faed5 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,10 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any + role. <command>CLUSTER</command> will skip over any tables that the calling user does not have permission to cluster. </para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..9bc97e1fc2 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2853,11 +2853,14 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); - if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, - relation->relname); + if (OidIsValid(table_oid)) + { + AclResult aclresult; + + aclresult = pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_INDEX, relation->relname); + } /* Lock heap before index to avoid deadlock. */ if (relId != oldRelId) @@ -3064,18 +3067,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..9b12bc44d7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -16986,6 +16986,7 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg) { char relkind; + AclResult aclresult; /* Nothing to do if the relation was not found. */ if (!OidIsValid(relId)) @@ -17006,36 +17007,9 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, - relation->relname); -} - -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; + aclresult = pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_TABLE, relation->relname); } /* diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 987b11d16c..11c23de8c1 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -721,17 +720,12 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, /*---------- * A role has privileges to vacuum or analyze the relation if any of the * following are true: - * - the role is a superuser - * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out index acfd9d1f4f..1473bc3175 100644 --- a/src/test/regress/expected/create_index.out +++ b/src/test/regress/expected/create_index.out @@ -2831,9 +2831,9 @@ RESET ROLE; GRANT USAGE ON SCHEMA pg_toast TO regress_reindexuser; SET SESSION ROLE regress_reindexuser; REINDEX TABLE pg_toast.pg_toast_1260; -ERROR: must be owner of table pg_toast_1260 +ERROR: permission denied for table pg_toast_1260 REINDEX INDEX pg_toast.pg_toast_1260_index; -ERROR: must be owner of index pg_toast_1260_index +ERROR: permission denied for index pg_toast_1260_index -- Clean up RESET ROLE; REVOKE USAGE ON SCHEMA pg_toast FROM regress_reindexuser; diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out index 3cf4ac8c9e..3e4dfcc2ec 100644 --- a/src/test/regress/expected/privileges.out +++ b/src/test/regress/expected/privileges.out @@ -2928,13 +2928,13 @@ WARNING: permission denied to analyze "maintain_test", skipping it VACUUM (ANALYZE) maintain_test; WARNING: permission denied to vacuum "maintain_test", skipping it CLUSTER maintain_test USING maintain_test_a_idx; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REFRESH MATERIALIZED VIEW refresh_test; -ERROR: must be owner of table refresh_test +ERROR: permission denied for table refresh_test REINDEX TABLE maintain_test; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REINDEX INDEX maintain_test_a_idx; -ERROR: must be owner of index maintain_test_a_idx +ERROR: permission denied for index maintain_test_a_idx REINDEX SCHEMA reindex_test; ERROR: must be owner of schema reindex_test RESET ROLE; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --/9DWx/yDrRhgMJTb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v4-0003-simplify-privilege-related-documentation-for-main.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v8 1/2] partial revert of ff9618e82a @ 2023-06-19 20:57 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-19 20:57 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +-- doc/src/sgml/ref/cluster.sgml | 5 +-- doc/src/sgml/ref/lock.sgml | 5 +-- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +-- src/backend/commands/cluster.c | 10 ++---- src/backend/commands/indexcmds.c | 27 +++++++-------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 34 +++---------------- src/backend/commands/vacuum.c | 8 +---- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++---- .../specs/cluster-conflict-partition.spec | 7 ++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/create_index.out | 4 +-- src/test/regress/expected/privileges.out | 8 ++--- src/test/regress/expected/vacuum.out | 18 ++++++++++ 17 files changed, 62 insertions(+), 106 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..f0dd7faed5 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,10 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any + role. <command>CLUSTER</command> will skip over any tables that the calling user does not have permission to cluster. </para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..9bc97e1fc2 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2853,11 +2853,14 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); - if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, - relation->relname); + if (OidIsValid(table_oid)) + { + AclResult aclresult; + + aclresult = pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_INDEX, relation->relname); + } /* Lock heap before index to avoid deadlock. */ if (relId != oldRelId) @@ -3064,18 +3067,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..9b12bc44d7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -16986,6 +16986,7 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg) { char relkind; + AclResult aclresult; /* Nothing to do if the relation was not found. */ if (!OidIsValid(relId)) @@ -17006,36 +17007,9 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, - relation->relname); -} - -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; + aclresult = pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_TABLE, relation->relname); } /* diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index bb79de4da6..e7a0450cc4 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -721,17 +720,12 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, /*---------- * A role has privileges to vacuum or analyze the relation if any of the * following are true: - * - the role is a superuser - * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out index acfd9d1f4f..1473bc3175 100644 --- a/src/test/regress/expected/create_index.out +++ b/src/test/regress/expected/create_index.out @@ -2831,9 +2831,9 @@ RESET ROLE; GRANT USAGE ON SCHEMA pg_toast TO regress_reindexuser; SET SESSION ROLE regress_reindexuser; REINDEX TABLE pg_toast.pg_toast_1260; -ERROR: must be owner of table pg_toast_1260 +ERROR: permission denied for table pg_toast_1260 REINDEX INDEX pg_toast.pg_toast_1260_index; -ERROR: must be owner of index pg_toast_1260_index +ERROR: permission denied for index pg_toast_1260_index -- Clean up RESET ROLE; REVOKE USAGE ON SCHEMA pg_toast FROM regress_reindexuser; diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out index 3cf4ac8c9e..3e4dfcc2ec 100644 --- a/src/test/regress/expected/privileges.out +++ b/src/test/regress/expected/privileges.out @@ -2928,13 +2928,13 @@ WARNING: permission denied to analyze "maintain_test", skipping it VACUUM (ANALYZE) maintain_test; WARNING: permission denied to vacuum "maintain_test", skipping it CLUSTER maintain_test USING maintain_test_a_idx; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REFRESH MATERIALIZED VIEW refresh_test; -ERROR: must be owner of table refresh_test +ERROR: permission denied for table refresh_test REINDEX TABLE maintain_test; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REINDEX INDEX maintain_test_a_idx; -ERROR: must be owner of index maintain_test_a_idx +ERROR: permission denied for index maintain_test_a_idx REINDEX SCHEMA reindex_test; ERROR: must be owner of schema reindex_test RESET ROLE; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --FL5UXtIhxfXey3p5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v8-0002-simplify-privilege-related-documentation-for-main.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v9 1/2] partial revert of ff9618e82a @ 2023-06-19 20:57 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-19 20:57 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +-- doc/src/sgml/ref/cluster.sgml | 5 +-- doc/src/sgml/ref/lock.sgml | 5 +-- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +-- src/backend/commands/cluster.c | 12 ++++--- src/backend/commands/indexcmds.c | 27 +++++++-------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 34 +++---------------- src/backend/commands/vacuum.c | 10 ++---- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++---- .../specs/cluster-conflict-partition.spec | 7 ++-- src/test/regress/expected/cluster.out | 7 +++- src/test/regress/expected/create_index.out | 4 +-- src/test/regress/expected/privileges.out | 8 ++--- src/test/regress/expected/vacuum.out | 18 ++++++++++ src/test/regress/sql/cluster.sql | 3 ++ 18 files changed, 74 insertions(+), 105 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..f0dd7faed5 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,10 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any + role. <command>CLUSTER</command> will skip over any tables that the calling user does not have permission to cluster. </para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..3bfabb6d10 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1694,10 +1694,13 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) continue; /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. + * It's possible that the user does not have privileges to CLUSTER the + * leaf partition despite having such privileges on the partitioned + * table. We skip any partitions which the user is not permitted to + * CLUSTER. */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1723,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..9bc97e1fc2 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2853,11 +2853,14 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); - if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, - relation->relname); + if (OidIsValid(table_oid)) + { + AclResult aclresult; + + aclresult = pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_INDEX, relation->relname); + } /* Lock heap before index to avoid deadlock. */ if (relId != oldRelId) @@ -3064,18 +3067,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..9b12bc44d7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -16986,6 +16986,7 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg) { char relkind; + AclResult aclresult; /* Nothing to do if the relation was not found. */ if (!OidIsValid(relId)) @@ -17006,36 +17007,9 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, - relation->relname); -} - -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; + aclresult = pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_TABLE, relation->relname); } /* diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index bb79de4da6..7fe6a54c06 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -721,17 +720,12 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, /*---------- * A role has privileges to vacuum or analyze the relation if any of the * following are true: - * - the role is a superuser - * - the role owns the relation * - the role owns the current database and the relation is not shared - * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors + * - the role has the MAINTAIN privilege on the relation *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..a13aafff0b 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -502,12 +502,17 @@ CREATE TABLE ptnowner1 PARTITION OF ptnowner FOR VALUES IN (1); CREATE ROLE regress_ptnowner; CREATE TABLE ptnowner2 PARTITION OF ptnowner FOR VALUES IN (2); ALTER TABLE ptnowner1 OWNER TO regress_ptnowner; +SET SESSION AUTHORIZATION regress_ptnowner; +CLUSTER ptnowner USING ptnowner_i_idx; +ERROR: permission denied for table ptnowner +RESET SESSION AUTHORIZATION; ALTER TABLE ptnowner OWNER TO regress_ptnowner; CREATE TEMP TABLE ptnowner_oldnodes AS SELECT oid, relname, relfilenode FROM pg_partition_tree('ptnowner') AS tree JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +520,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out index acfd9d1f4f..1473bc3175 100644 --- a/src/test/regress/expected/create_index.out +++ b/src/test/regress/expected/create_index.out @@ -2831,9 +2831,9 @@ RESET ROLE; GRANT USAGE ON SCHEMA pg_toast TO regress_reindexuser; SET SESSION ROLE regress_reindexuser; REINDEX TABLE pg_toast.pg_toast_1260; -ERROR: must be owner of table pg_toast_1260 +ERROR: permission denied for table pg_toast_1260 REINDEX INDEX pg_toast.pg_toast_1260_index; -ERROR: must be owner of index pg_toast_1260_index +ERROR: permission denied for index pg_toast_1260_index -- Clean up RESET ROLE; REVOKE USAGE ON SCHEMA pg_toast FROM regress_reindexuser; diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out index 3cf4ac8c9e..3e4dfcc2ec 100644 --- a/src/test/regress/expected/privileges.out +++ b/src/test/regress/expected/privileges.out @@ -2928,13 +2928,13 @@ WARNING: permission denied to analyze "maintain_test", skipping it VACUUM (ANALYZE) maintain_test; WARNING: permission denied to vacuum "maintain_test", skipping it CLUSTER maintain_test USING maintain_test_a_idx; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REFRESH MATERIALIZED VIEW refresh_test; -ERROR: must be owner of table refresh_test +ERROR: permission denied for table refresh_test REINDEX TABLE maintain_test; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REINDEX INDEX maintain_test_a_idx; -ERROR: must be owner of index maintain_test_a_idx +ERROR: permission denied for index maintain_test_a_idx REINDEX SCHEMA reindex_test; ERROR: must be owner of schema reindex_test RESET ROLE; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; diff --git a/src/test/regress/sql/cluster.sql b/src/test/regress/sql/cluster.sql index a4cfaae807..b7115f8610 100644 --- a/src/test/regress/sql/cluster.sql +++ b/src/test/regress/sql/cluster.sql @@ -238,6 +238,9 @@ CREATE TABLE ptnowner1 PARTITION OF ptnowner FOR VALUES IN (1); CREATE ROLE regress_ptnowner; CREATE TABLE ptnowner2 PARTITION OF ptnowner FOR VALUES IN (2); ALTER TABLE ptnowner1 OWNER TO regress_ptnowner; +SET SESSION AUTHORIZATION regress_ptnowner; +CLUSTER ptnowner USING ptnowner_i_idx; +RESET SESSION AUTHORIZATION; ALTER TABLE ptnowner OWNER TO regress_ptnowner; CREATE TEMP TABLE ptnowner_oldnodes AS SELECT oid, relname, relfilenode FROM pg_partition_tree('ptnowner') AS tree -- 2.25.1 --qMm9M+Fa2AknHoGS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v9-0002-simplify-privilege-related-documentation-for-main.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v4 2/3] partial revert of ff9618e82a @ 2023-06-19 20:57 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-19 20:57 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +-- doc/src/sgml/ref/cluster.sgml | 5 +-- doc/src/sgml/ref/lock.sgml | 5 +-- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +-- src/backend/commands/cluster.c | 10 ++---- src/backend/commands/indexcmds.c | 27 +++++++-------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 34 +++---------------- src/backend/commands/vacuum.c | 8 +---- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++---- .../specs/cluster-conflict-partition.spec | 7 ++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/create_index.out | 4 +-- src/test/regress/expected/privileges.out | 8 ++--- src/test/regress/expected/vacuum.out | 18 ++++++++++ 17 files changed, 62 insertions(+), 106 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..f0dd7faed5 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,10 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any + role. <command>CLUSTER</command> will skip over any tables that the calling user does not have permission to cluster. </para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..9bc97e1fc2 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2853,11 +2853,14 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); - if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, - relation->relname); + if (OidIsValid(table_oid)) + { + AclResult aclresult; + + aclresult = pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_INDEX, relation->relname); + } /* Lock heap before index to avoid deadlock. */ if (relId != oldRelId) @@ -3064,18 +3067,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..9b12bc44d7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -16986,6 +16986,7 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg) { char relkind; + AclResult aclresult; /* Nothing to do if the relation was not found. */ if (!OidIsValid(relId)) @@ -17006,36 +17007,9 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, - relation->relname); -} - -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; + aclresult = pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_TABLE, relation->relname); } /* diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 987b11d16c..11c23de8c1 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -721,17 +720,12 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, /*---------- * A role has privileges to vacuum or analyze the relation if any of the * following are true: - * - the role is a superuser - * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out index acfd9d1f4f..1473bc3175 100644 --- a/src/test/regress/expected/create_index.out +++ b/src/test/regress/expected/create_index.out @@ -2831,9 +2831,9 @@ RESET ROLE; GRANT USAGE ON SCHEMA pg_toast TO regress_reindexuser; SET SESSION ROLE regress_reindexuser; REINDEX TABLE pg_toast.pg_toast_1260; -ERROR: must be owner of table pg_toast_1260 +ERROR: permission denied for table pg_toast_1260 REINDEX INDEX pg_toast.pg_toast_1260_index; -ERROR: must be owner of index pg_toast_1260_index +ERROR: permission denied for index pg_toast_1260_index -- Clean up RESET ROLE; REVOKE USAGE ON SCHEMA pg_toast FROM regress_reindexuser; diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out index 3cf4ac8c9e..3e4dfcc2ec 100644 --- a/src/test/regress/expected/privileges.out +++ b/src/test/regress/expected/privileges.out @@ -2928,13 +2928,13 @@ WARNING: permission denied to analyze "maintain_test", skipping it VACUUM (ANALYZE) maintain_test; WARNING: permission denied to vacuum "maintain_test", skipping it CLUSTER maintain_test USING maintain_test_a_idx; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REFRESH MATERIALIZED VIEW refresh_test; -ERROR: must be owner of table refresh_test +ERROR: permission denied for table refresh_test REINDEX TABLE maintain_test; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REINDEX INDEX maintain_test_a_idx; -ERROR: must be owner of index maintain_test_a_idx +ERROR: permission denied for index maintain_test_a_idx REINDEX SCHEMA reindex_test; ERROR: must be owner of schema reindex_test RESET ROLE; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --/9DWx/yDrRhgMJTb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v4-0003-simplify-privilege-related-documentation-for-main.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v7 1/2] partial revert of ff9618e82a @ 2023-06-19 20:57 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-19 20:57 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +-- doc/src/sgml/ref/cluster.sgml | 5 +-- doc/src/sgml/ref/lock.sgml | 5 +-- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +-- src/backend/commands/cluster.c | 10 ++---- src/backend/commands/indexcmds.c | 27 +++++++-------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 34 +++---------------- src/backend/commands/vacuum.c | 8 +---- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++---- .../specs/cluster-conflict-partition.spec | 7 ++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/create_index.out | 4 +-- src/test/regress/expected/privileges.out | 8 ++--- src/test/regress/expected/vacuum.out | 18 ++++++++++ 17 files changed, 62 insertions(+), 106 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..f0dd7faed5 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,10 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any + role. <command>CLUSTER</command> will skip over any tables that the calling user does not have permission to cluster. </para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..9bc97e1fc2 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2853,11 +2853,14 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); - if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, - relation->relname); + if (OidIsValid(table_oid)) + { + AclResult aclresult; + + aclresult = pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_INDEX, relation->relname); + } /* Lock heap before index to avoid deadlock. */ if (relId != oldRelId) @@ -3064,18 +3067,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..9b12bc44d7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -16986,6 +16986,7 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg) { char relkind; + AclResult aclresult; /* Nothing to do if the relation was not found. */ if (!OidIsValid(relId)) @@ -17006,36 +17007,9 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, - relation->relname); -} - -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; + aclresult = pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_TABLE, relation->relname); } /* diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index bb79de4da6..e7a0450cc4 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -721,17 +720,12 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, /*---------- * A role has privileges to vacuum or analyze the relation if any of the * following are true: - * - the role is a superuser - * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out index acfd9d1f4f..1473bc3175 100644 --- a/src/test/regress/expected/create_index.out +++ b/src/test/regress/expected/create_index.out @@ -2831,9 +2831,9 @@ RESET ROLE; GRANT USAGE ON SCHEMA pg_toast TO regress_reindexuser; SET SESSION ROLE regress_reindexuser; REINDEX TABLE pg_toast.pg_toast_1260; -ERROR: must be owner of table pg_toast_1260 +ERROR: permission denied for table pg_toast_1260 REINDEX INDEX pg_toast.pg_toast_1260_index; -ERROR: must be owner of index pg_toast_1260_index +ERROR: permission denied for index pg_toast_1260_index -- Clean up RESET ROLE; REVOKE USAGE ON SCHEMA pg_toast FROM regress_reindexuser; diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out index 3cf4ac8c9e..3e4dfcc2ec 100644 --- a/src/test/regress/expected/privileges.out +++ b/src/test/regress/expected/privileges.out @@ -2928,13 +2928,13 @@ WARNING: permission denied to analyze "maintain_test", skipping it VACUUM (ANALYZE) maintain_test; WARNING: permission denied to vacuum "maintain_test", skipping it CLUSTER maintain_test USING maintain_test_a_idx; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REFRESH MATERIALIZED VIEW refresh_test; -ERROR: must be owner of table refresh_test +ERROR: permission denied for table refresh_test REINDEX TABLE maintain_test; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REINDEX INDEX maintain_test_a_idx; -ERROR: must be owner of index maintain_test_a_idx +ERROR: permission denied for index maintain_test_a_idx REINDEX SCHEMA reindex_test; ERROR: must be owner of schema reindex_test RESET ROLE; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --/9DWx/yDrRhgMJTb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0002-simplify-privilege-related-documentation-for-main.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v8 1/2] partial revert of ff9618e82a @ 2023-06-19 20:57 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-19 20:57 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +-- doc/src/sgml/ref/cluster.sgml | 5 +-- doc/src/sgml/ref/lock.sgml | 5 +-- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +-- src/backend/commands/cluster.c | 10 ++---- src/backend/commands/indexcmds.c | 27 +++++++-------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 34 +++---------------- src/backend/commands/vacuum.c | 8 +---- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++---- .../specs/cluster-conflict-partition.spec | 7 ++-- src/test/regress/expected/cluster.out | 3 +- src/test/regress/expected/create_index.out | 4 +-- src/test/regress/expected/privileges.out | 8 ++--- src/test/regress/expected/vacuum.out | 18 ++++++++++ 17 files changed, 62 insertions(+), 106 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..f0dd7faed5 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,10 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any + role. <command>CLUSTER</command> will skip over any tables that the calling user does not have permission to cluster. </para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..38834356b9 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1693,11 +1693,8 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) if (get_rel_relkind(indexrelid) != RELKIND_INDEX) continue; - /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. - */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1717,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..9bc97e1fc2 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2853,11 +2853,14 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); - if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, - relation->relname); + if (OidIsValid(table_oid)) + { + AclResult aclresult; + + aclresult = pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_INDEX, relation->relname); + } /* Lock heap before index to avoid deadlock. */ if (relId != oldRelId) @@ -3064,18 +3067,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..9b12bc44d7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -16986,6 +16986,7 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg) { char relkind; + AclResult aclresult; /* Nothing to do if the relation was not found. */ if (!OidIsValid(relId)) @@ -17006,36 +17007,9 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, - relation->relname); -} - -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; + aclresult = pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_TABLE, relation->relname); } /* diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index bb79de4da6..e7a0450cc4 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -721,17 +720,12 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, /*---------- * A role has privileges to vacuum or analyze the relation if any of the * following are true: - * - the role is a superuser - * - the role owns the relation * - the role owns the current database and the relation is not shared * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..27a5dff5d4 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -508,6 +508,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +516,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out index acfd9d1f4f..1473bc3175 100644 --- a/src/test/regress/expected/create_index.out +++ b/src/test/regress/expected/create_index.out @@ -2831,9 +2831,9 @@ RESET ROLE; GRANT USAGE ON SCHEMA pg_toast TO regress_reindexuser; SET SESSION ROLE regress_reindexuser; REINDEX TABLE pg_toast.pg_toast_1260; -ERROR: must be owner of table pg_toast_1260 +ERROR: permission denied for table pg_toast_1260 REINDEX INDEX pg_toast.pg_toast_1260_index; -ERROR: must be owner of index pg_toast_1260_index +ERROR: permission denied for index pg_toast_1260_index -- Clean up RESET ROLE; REVOKE USAGE ON SCHEMA pg_toast FROM regress_reindexuser; diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out index 3cf4ac8c9e..3e4dfcc2ec 100644 --- a/src/test/regress/expected/privileges.out +++ b/src/test/regress/expected/privileges.out @@ -2928,13 +2928,13 @@ WARNING: permission denied to analyze "maintain_test", skipping it VACUUM (ANALYZE) maintain_test; WARNING: permission denied to vacuum "maintain_test", skipping it CLUSTER maintain_test USING maintain_test_a_idx; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REFRESH MATERIALIZED VIEW refresh_test; -ERROR: must be owner of table refresh_test +ERROR: permission denied for table refresh_test REINDEX TABLE maintain_test; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REINDEX INDEX maintain_test_a_idx; -ERROR: must be owner of index maintain_test_a_idx +ERROR: permission denied for index maintain_test_a_idx REINDEX SCHEMA reindex_test; ERROR: must be owner of schema reindex_test RESET ROLE; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; -- 2.25.1 --FL5UXtIhxfXey3p5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v8-0002-simplify-privilege-related-documentation-for-main.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v9 1/2] partial revert of ff9618e82a @ 2023-06-19 20:57 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Nathan Bossart @ 2023-06-19 20:57 UTC (permalink / raw) --- doc/src/sgml/ref/analyze.sgml | 5 +-- doc/src/sgml/ref/cluster.sgml | 5 +-- doc/src/sgml/ref/lock.sgml | 5 +-- doc/src/sgml/ref/reindex.sgml | 6 +--- doc/src/sgml/ref/vacuum.sgml | 5 +-- src/backend/commands/cluster.c | 12 ++++--- src/backend/commands/indexcmds.c | 27 +++++++-------- src/backend/commands/lockcmds.c | 8 ----- src/backend/commands/tablecmds.c | 34 +++---------------- src/backend/commands/vacuum.c | 10 ++---- src/include/commands/tablecmds.h | 1 - .../expected/cluster-conflict-partition.out | 14 ++++---- .../specs/cluster-conflict-partition.spec | 7 ++-- src/test/regress/expected/cluster.out | 7 +++- src/test/regress/expected/create_index.out | 4 +-- src/test/regress/expected/privileges.out | 8 ++--- src/test/regress/expected/vacuum.out | 18 ++++++++++ src/test/regress/sql/cluster.sql | 3 ++ 18 files changed, 74 insertions(+), 105 deletions(-) diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml index 20c6f9939f..30a893230e 100644 --- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml @@ -190,10 +190,7 @@ ANALYZE [ VERBOSE ] [ <replaceable class="parameter">table_and_columns</replacea analyze all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>ANALYZE</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>ANALYZE</command> a partitioned table, it is also - permitted to <command>ANALYZE</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>ANALYZE</command> will skip over any tables that the calling user does not have permission to analyze. </para> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 29f0f1fd90..f0dd7faed5 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -137,10 +137,7 @@ CLUSTER [VERBOSE] on the table or be the table's owner, a superuser, or a role with privileges of the <link linkend="predefined-roles-table"><literal>pg_maintain</literal></link> - role. If a role has permission to <command>CLUSTER</command> a partitioned - table, it is also permitted to <command>CLUSTER</command> each of its - partitions, regardless of whether the role has the aforementioned - privileges on the partition. <command>CLUSTER</command> will skip over any + role. <command>CLUSTER</command> will skip over any tables that the calling user does not have permission to cluster. </para> diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml index 5b3b2b793a..8524182211 100644 --- a/doc/src/sgml/ref/lock.sgml +++ b/doc/src/sgml/ref/lock.sgml @@ -177,10 +177,7 @@ LOCK [ TABLE ] [ ONLY ] <replaceable class="parameter">name</replaceable> [ * ] MODE</literal> (or a less-conflicting mode as described in <xref linkend="explicit-locking"/>) is permitted. If a user has <literal>SELECT</literal> privileges on the table, <literal>ACCESS SHARE - MODE</literal> is permitted. If a role has permission to lock a - partitioned table, it is also permitted to lock each of its partitions, - regardless of whether the role has the aforementioned privileges on the - partition. + MODE</literal> is permitted. </para> <para> diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 71455dfdc7..23f8c7630b 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -306,11 +306,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { DA indexes on shared catalogs will be skipped unless the user owns the catalog (which typically won't be the case), has privileges of the <literal>pg_maintain</literal> role, or has the <literal>MAINTAIN</literal> - privilege on the catalog. If a role has permission to - <command>REINDEX</command> a partitioned table, it is also permitted to - <command>REINDEX</command> each of its partitions, regardless of whether the - role has the aforementioned privileges on the partition. Of course, - superusers can always reindex anything. + privilege on the catalog. Of course, superusers can always reindex anything. </para> <para> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 57bc4c23ec..445325e14c 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -452,10 +452,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet vacuum all tables in their databases, except shared catalogs. (The restriction for shared catalogs means that a true database-wide <command>VACUUM</command> can only be performed by superusers and roles - with privileges of <literal>pg_maintain</literal>.) If a role has - permission to <command>VACUUM</command> a partitioned table, it is also - permitted to <command>VACUUM</command> each of its partitions, regardless - of whether the role has the aforementioned privileges on the partition. + with privileges of <literal>pg_maintain</literal>.) <command>VACUUM</command> will skip over any tables that the calling user does not have permission to vacuum. </para> diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index 369fea7c04..3bfabb6d10 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -1694,10 +1694,13 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) continue; /* - * We already checked that the user has privileges to CLUSTER the - * partitioned table when we locked it earlier, so there's no need to - * check the privileges again here. + * It's possible that the user does not have privileges to CLUSTER the + * leaf partition despite having such privileges on the partitioned + * table. We skip any partitions which the user is not permitted to + * CLUSTER. */ + if (!cluster_is_permitted_for_relation(relid, GetUserId())) + continue; /* Use a permanent memory context for the result list */ old_context = MemoryContextSwitchTo(cluster_context); @@ -1720,8 +1723,7 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid) static bool cluster_is_permitted_for_relation(Oid relid, Oid userid) { - if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, userid, ACL_MAINTAIN)) + if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK) return true; ereport(WARNING, diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index a5168c9f09..9bc97e1fc2 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -2853,11 +2853,14 @@ RangeVarCallbackForReindexIndex(const RangeVar *relation, /* Check permissions */ table_oid = IndexGetRelation(relId, true); - if (OidIsValid(table_oid) && - pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(table_oid, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_INDEX, - relation->relname); + if (OidIsValid(table_oid)) + { + AclResult aclresult; + + aclresult = pg_class_aclcheck(table_oid, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_INDEX, relation->relname); + } /* Lock heap before index to avoid deadlock. */ if (relId != oldRelId) @@ -3064,18 +3067,12 @@ ReindexMultipleTables(const char *objectName, ReindexObjectType objectKind, continue; /* - * The table can be reindexed if the user has been granted MAINTAIN on - * the table or one of its partition ancestors or the user is a - * superuser, the table owner, or the database/schema owner (but in - * the latter case, only if it's not a shared relation). - * pg_class_aclcheck includes the superuser case, and depending on - * objectKind we already know that the user has permission to run - * REINDEX on this database or schema per the permission checks at the - * beginning of this routine. + * We already checked privileges on the database or schema, but we + * further restrict reindexing shared catalogs to roles with the + * MAINTAIN privilege on the relation. */ if (classtuple->relisshared && - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK) continue; /* diff --git a/src/backend/commands/lockcmds.c b/src/backend/commands/lockcmds.c index 43c7d7f4bb..92662cbbc8 100644 --- a/src/backend/commands/lockcmds.c +++ b/src/backend/commands/lockcmds.c @@ -19,7 +19,6 @@ #include "catalog/namespace.h" #include "catalog/pg_inherits.h" #include "commands/lockcmds.h" -#include "commands/tablecmds.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "parser/parse_clause.h" @@ -297,12 +296,5 @@ LockTableAclCheck(Oid reloid, LOCKMODE lockmode, Oid userid) aclresult = pg_class_aclcheck(reloid, userid, aclmask); - /* - * If this is a partition, check permissions of its ancestors if needed. - */ - if (aclresult != ACLCHECK_OK && - has_partition_ancestor_privs(reloid, userid, ACL_MAINTAIN)) - aclresult = ACLCHECK_OK; - return aclresult; } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 4d49d70c33..9b12bc44d7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -16986,6 +16986,7 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg) { char relkind; + AclResult aclresult; /* Nothing to do if the relation was not found. */ if (!OidIsValid(relId)) @@ -17006,36 +17007,9 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation, errmsg("\"%s\" is not a table or materialized view", relation->relname))); /* Check permissions */ - if (pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN) != ACLCHECK_OK && - !has_partition_ancestor_privs(relId, GetUserId(), ACL_MAINTAIN)) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, - relation->relname); -} - -/* - * If relid is a partition, returns whether userid has any of the privileges - * specified in acl on any of its ancestors. Otherwise, returns false. - */ -bool -has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl) -{ - List *ancestors; - ListCell *lc; - - if (!get_rel_relispartition(relid)) - return false; - - ancestors = get_partition_ancestors(relid); - foreach(lc, ancestors) - { - Oid ancestor = lfirst_oid(lc); - - if (OidIsValid(ancestor) && - pg_class_aclcheck(ancestor, userid, acl) == ACLCHECK_OK) - return true; - } - - return false; + aclresult = pg_class_aclcheck(relId, GetUserId(), ACL_MAINTAIN); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_TABLE, relation->relname); } /* diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index bb79de4da6..7fe6a54c06 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -41,7 +41,6 @@ #include "catalog/pg_namespace.h" #include "commands/cluster.h" #include "commands/defrem.h" -#include "commands/tablecmds.h" #include "commands/vacuum.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -721,17 +720,12 @@ vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple, /*---------- * A role has privileges to vacuum or analyze the relation if any of the * following are true: - * - the role is a superuser - * - the role owns the relation * - the role owns the current database and the relation is not shared - * - the role has been granted the MAINTAIN privilege on the relation - * - the role has privileges to vacuum/analyze any of the relation's - * partition ancestors + * - the role has the MAINTAIN privilege on the relation *---------- */ if ((object_ownercheck(DatabaseRelationId, MyDatabaseId, GetUserId()) && !reltuple->relisshared) || - pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK || - has_partition_ancestor_privs(relid, GetUserId(), ACL_MAINTAIN)) + pg_class_aclcheck(relid, GetUserId(), ACL_MAINTAIN) == ACLCHECK_OK) return true; relname = NameStr(reltuple->relname); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 17b9404937..250d89ff88 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -99,7 +99,6 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit, extern void RangeVarCallbackMaintainsTable(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); -extern bool has_partition_ancestor_privs(Oid relid, Oid userid, AclMode acl); extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, Oid relId, Oid oldRelId, void *arg); diff --git a/src/test/isolation/expected/cluster-conflict-partition.out b/src/test/isolation/expected/cluster-conflict-partition.out index 8d21276996..7be9e56ef1 100644 --- a/src/test/isolation/expected/cluster-conflict-partition.out +++ b/src/test/isolation/expected/cluster-conflict-partition.out @@ -3,7 +3,7 @@ Parsed test spec with 2 sessions starting permutation: s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; step s2_cluster: <... completed> @@ -11,7 +11,7 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_parent: LOCK cluster_part_tab IN SHARE UPDATE EXCLUSIVE MODE; step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> step s1_commit: COMMIT; @@ -21,17 +21,15 @@ step s2_reset: RESET ROLE; starting permutation: s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset step s1_begin: BEGIN; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_auth: SET ROLE regress_cluster_part; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; starting permutation: s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset step s1_begin: BEGIN; -step s2_auth: SET ROLE regress_cluster_part; +step s2_auth: SET ROLE regress_cluster_part; SET client_min_messages = ERROR; step s1_lock_child: LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; <waiting ...> +step s2_cluster: CLUSTER cluster_part_tab USING cluster_part_ind; step s1_commit: COMMIT; -step s2_cluster: <... completed> step s2_reset: RESET ROLE; diff --git a/src/test/isolation/specs/cluster-conflict-partition.spec b/src/test/isolation/specs/cluster-conflict-partition.spec index ae38cb4ee3..4d38a7f49a 100644 --- a/src/test/isolation/specs/cluster-conflict-partition.spec +++ b/src/test/isolation/specs/cluster-conflict-partition.spec @@ -23,12 +23,15 @@ step s1_lock_child { LOCK cluster_part_tab1 IN SHARE UPDATE EXCLUSIVE MODE; step s1_commit { COMMIT; } session s2 -step s2_auth { SET ROLE regress_cluster_part; } +step s2_auth { SET ROLE regress_cluster_part; SET client_min_messages = ERROR; } step s2_cluster { CLUSTER cluster_part_tab USING cluster_part_ind; } step s2_reset { RESET ROLE; } -# CLUSTER waits if locked, passes for all cases. +# CLUSTER on the parent waits if locked, passes for all cases. permutation s1_begin s1_lock_parent s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_parent s2_cluster s1_commit s2_reset + +# When taking a lock on a partition leaf, CLUSTER on the parent skips +# the leaf, passes for all cases. permutation s1_begin s1_lock_child s2_auth s2_cluster s1_commit s2_reset permutation s1_begin s2_auth s1_lock_child s2_cluster s1_commit s2_reset diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 2eec483eaa..a13aafff0b 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -502,12 +502,17 @@ CREATE TABLE ptnowner1 PARTITION OF ptnowner FOR VALUES IN (1); CREATE ROLE regress_ptnowner; CREATE TABLE ptnowner2 PARTITION OF ptnowner FOR VALUES IN (2); ALTER TABLE ptnowner1 OWNER TO regress_ptnowner; +SET SESSION AUTHORIZATION regress_ptnowner; +CLUSTER ptnowner USING ptnowner_i_idx; +ERROR: permission denied for table ptnowner +RESET SESSION AUTHORIZATION; ALTER TABLE ptnowner OWNER TO regress_ptnowner; CREATE TEMP TABLE ptnowner_oldnodes AS SELECT oid, relname, relfilenode FROM pg_partition_tree('ptnowner') AS tree JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +WARNING: permission denied to cluster "ptnowner2", skipping it RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -515,7 +520,7 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a -----------+---------- ptnowner | t ptnowner1 | f - ptnowner2 | f + ptnowner2 | t (3 rows) DROP TABLE ptnowner; diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out index acfd9d1f4f..1473bc3175 100644 --- a/src/test/regress/expected/create_index.out +++ b/src/test/regress/expected/create_index.out @@ -2831,9 +2831,9 @@ RESET ROLE; GRANT USAGE ON SCHEMA pg_toast TO regress_reindexuser; SET SESSION ROLE regress_reindexuser; REINDEX TABLE pg_toast.pg_toast_1260; -ERROR: must be owner of table pg_toast_1260 +ERROR: permission denied for table pg_toast_1260 REINDEX INDEX pg_toast.pg_toast_1260_index; -ERROR: must be owner of index pg_toast_1260_index +ERROR: permission denied for index pg_toast_1260_index -- Clean up RESET ROLE; REVOKE USAGE ON SCHEMA pg_toast FROM regress_reindexuser; diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out index 3cf4ac8c9e..3e4dfcc2ec 100644 --- a/src/test/regress/expected/privileges.out +++ b/src/test/regress/expected/privileges.out @@ -2928,13 +2928,13 @@ WARNING: permission denied to analyze "maintain_test", skipping it VACUUM (ANALYZE) maintain_test; WARNING: permission denied to vacuum "maintain_test", skipping it CLUSTER maintain_test USING maintain_test_a_idx; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REFRESH MATERIALIZED VIEW refresh_test; -ERROR: must be owner of table refresh_test +ERROR: permission denied for table refresh_test REINDEX TABLE maintain_test; -ERROR: must be owner of table maintain_test +ERROR: permission denied for table maintain_test REINDEX INDEX maintain_test_a_idx; -ERROR: must be owner of index maintain_test_a_idx +ERROR: permission denied for index maintain_test_a_idx REINDEX SCHEMA reindex_test; ERROR: must be owner of schema reindex_test RESET ROLE; diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 41e020cf20..4def90b805 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -442,14 +442,20 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO regress_vacuum; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; -- Only one partition owned by other user. ALTER TABLE vacowned_parted OWNER TO CURRENT_USER; @@ -478,14 +484,26 @@ ALTER TABLE vacowned_parted OWNER TO regress_vacuum; ALTER TABLE vacowned_part1 OWNER TO CURRENT_USER; SET ROLE regress_vacuum; VACUUM vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it ANALYZE vacowned_parted; +WARNING: permission denied to analyze "vacowned_part1", skipping it +WARNING: permission denied to analyze "vacowned_part2", skipping it ANALYZE vacowned_part1; +WARNING: permission denied to analyze "vacowned_part1", skipping it ANALYZE vacowned_part2; +WARNING: permission denied to analyze "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_parted; +WARNING: permission denied to vacuum "vacowned_part1", skipping it +WARNING: permission denied to vacuum "vacowned_part2", skipping it VACUUM (ANALYZE) vacowned_part1; +WARNING: permission denied to vacuum "vacowned_part1", skipping it VACUUM (ANALYZE) vacowned_part2; +WARNING: permission denied to vacuum "vacowned_part2", skipping it RESET ROLE; DROP TABLE vacowned; DROP TABLE vacowned_parted; diff --git a/src/test/regress/sql/cluster.sql b/src/test/regress/sql/cluster.sql index a4cfaae807..b7115f8610 100644 --- a/src/test/regress/sql/cluster.sql +++ b/src/test/regress/sql/cluster.sql @@ -238,6 +238,9 @@ CREATE TABLE ptnowner1 PARTITION OF ptnowner FOR VALUES IN (1); CREATE ROLE regress_ptnowner; CREATE TABLE ptnowner2 PARTITION OF ptnowner FOR VALUES IN (2); ALTER TABLE ptnowner1 OWNER TO regress_ptnowner; +SET SESSION AUTHORIZATION regress_ptnowner; +CLUSTER ptnowner USING ptnowner_i_idx; +RESET SESSION AUTHORIZATION; ALTER TABLE ptnowner OWNER TO regress_ptnowner; CREATE TEMP TABLE ptnowner_oldnodes AS SELECT oid, relname, relfilenode FROM pg_partition_tree('ptnowner') AS tree -- 2.25.1 --qMm9M+Fa2AknHoGS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v9-0002-simplify-privilege-related-documentation-for-main.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-01-02 20:15 Masahiko Sawada <[email protected]> 0 siblings, 1 reply; 35+ messages in thread From: Masahiko Sawada @ 2026-01-02 20:15 UTC (permalink / raw) To: Matheus Alcantara <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers Sorry for the late reply. On Thu, Dec 11, 2025 at 4:03 AM Matheus Alcantara <[email protected]> wrote: > > I've spent some more time on this patch cleaning up some things and > trying to simplify some things. > > I've renamed "copy_for_batch_insert_threshold" to > "batch_with_copy_threshold" and removed the boolean option > "use_copy_for_batch_insert", so now to enable the COPY usage for batch > inserts it only need to set batch_with_copy_threshold to a number > greater than 0. > > Also the COPY can only be used if batching is also enabled (batch_size > > 1) and it will only be used for the COPY FROM on a foreign table and for > inserts into table partitions that are also foreign tables. Thank you for updating the patch! + + /* + * Set batch_with_copy_threshold from foreign server/table options. We do + * this outside of create_foreign_modify() because we only want to use + * COPY as a remote SQL when a COPY FROM on a foreign table is executed or + * an insert is being performed on a table partition. In both cases the + * BeginForeignInsert fdw routine is called. + */ + fmstate->batch_with_copy_threshold = get_batch_with_copy_threshold(rel); Does it mean that we could end up using the COPY method not only when executing COPY FROM but also when executing INSERT with tuple routings? If so, how does the EXPLAIN command show the remote SQL? Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-01-02 20:33 Matheus Alcantara <[email protected]> parent: Masahiko Sawada <[email protected]> 0 siblings, 2 replies; 35+ messages in thread From: Matheus Alcantara @ 2026-01-02 20:33 UTC (permalink / raw) To: Masahiko Sawada <[email protected]>; Matheus Alcantara <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers On Fri Jan 2, 2026 at 5:15 PM -03, Masahiko Sawada wrote: > + > + /* > + * Set batch_with_copy_threshold from foreign server/table options. We do > + * this outside of create_foreign_modify() because we only want to use > + * COPY as a remote SQL when a COPY FROM on a foreign table is executed or > + * an insert is being performed on a table partition. In both cases the > + * BeginForeignInsert fdw routine is called. > + */ > + fmstate->batch_with_copy_threshold = get_batch_with_copy_threshold(rel); > > Does it mean that we could end up using the COPY method not only when > executing COPY FROM but also when executing INSERT with tuple > routings? If so, how does the EXPLAIN command show the remote SQL? > It meas that we could also use the COPY method to insert rows into a specific table partition that is a foreign table. Let's say that an user execute an INSERT INTO on a partitioned table that has partitions that are postgres_fdw tables, with this patch we could use the COPY method to insert the rows on these partitions. On this scenario we would not have issue with EXPLAIN output because currently we do not show the remote SQL being executed on each partition that is involved on the INSERT statement. If an user execute an INSERT directly into a postgres_fdw table we will use the normal INSERT statement as we use today. Thanks for taking a look at this. -- Matheus Alcantara EDB: https://www.enterprisedb.com ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-01-03 12:45 Dewei Dai <[email protected]> parent: Matheus Alcantara <[email protected]> 1 sibling, 1 reply; 35+ messages in thread From: Dewei Dai @ 2026-01-03 12:45 UTC (permalink / raw) To: Matheus Alcantara <[email protected]>; Masahiko Sawada <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers Hi Matheus, I just reviewed the v8 patch and got a few comments: 1 - in function `execute_foreign_modify_using_copy` The `res` object obtained from the first call to `pgfdw_get_result` is not freed, maybe you can use `PQclear` to release it 2 - in function `execute_foreign_modify_using_copy` After using `copy_data`,it appears it can be released by calling `destroyStringInfo`. 3 - in function `convert_slot_to_copy_text` The value returned by the OutputFunctionCall function can be freed by calling pfree 4 - in function `execute_foreign_modify_using_copy` ```Send initial COPY data if the buffer reach the limit to avoid large ``` Typo: reach -> reaches Best regards， Dewei Dai [email protected] From: Matheus Alcantara Date: 2026-01-03 04:33 To: Masahiko Sawada; Matheus Alcantara CC: Andrew Dunstan; jian he; Tomas Vondra; [email protected] Subject: Re: postgres_fdw: Use COPY to speed up batch inserts On Fri Jan 2, 2026 at 5:15 PM -03, Masahiko Sawada wrote: > + > + /* > + * Set batch_with_copy_threshold from foreign server/table options. We do > + * this outside of create_foreign_modify() because we only want to use > + * COPY as a remote SQL when a COPY FROM on a foreign table is executed or > + * an insert is being performed on a table partition. In both cases the > + * BeginForeignInsert fdw routine is called. > + */ > + fmstate->batch_with_copy_threshold = get_batch_with_copy_threshold(rel); > > Does it mean that we could end up using the COPY method not only when > executing COPY FROM but also when executing INSERT with tuple > routings? If so, how does the EXPLAIN command show the remote SQL? > It meas that we could also use the COPY method to insert rows into a specific table partition that is a foreign table. Let's say that an user execute an INSERT INTO on a partitioned table that has partitions that are postgres_fdw tables, with this patch we could use the COPY method to insert the rows on these partitions. On this scenario we would not have issue with EXPLAIN output because currently we do not show the remote SQL being executed on each partition that is involved on the INSERT statement. If an user execute an INSERT directly into a postgres_fdw table we will use the normal INSERT statement as we use today. Thanks for taking a look at this. -- Matheus Alcantara EDB: https://www.enterprisedb.com ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-01-05 13:43 Matheus Alcantara <[email protected]> parent: Dewei Dai <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Matheus Alcantara @ 2026-01-05 13:43 UTC (permalink / raw) To: Dewei Dai <[email protected]>; Matheus Alcantara <[email protected]>; Masahiko Sawada <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers Hi, thank you for reviewing this patch! On Sat Jan 3, 2026 at 9:45 AM -03, Dewei Dai wrote: > 1 - in function `execute_foreign_modify_using_copy` > The `res` object obtained from the first call to `pgfdw_get_result` > is not freed, maybe you can use `PQclear` to release it > Fixed. > 2 - in function `execute_foreign_modify_using_copy` > After using `copy_data`,it appears it can be released by > calling `destroyStringInfo`. > I'm wondering if we should call destroyStringInfo(©_data) or pfree(copy_data.data). Other functions use the pfree version, so I decided to use the same. (I actually tried to use destroyStringInfo() but the postgres_fdw tests kept running for longer than usual, so I think that using the pfree is correct) > 3 - in function `convert_slot_to_copy_text` > The value returned by the OutputFunctionCall function can be > freed by calling pfree > We have other calls to OutputFunctionCall() on postgres_fdw.c and I'm not seeing a subsequent call to pfree. IIUC the returned valued will be allocated on the current memory context which will be free at the end of query execution, so I don't think that a pfree here is necessary, or I'm missing something? > 4 - in function `execute_foreign_modify_using_copy` > ```Send initial COPY data if the buffer reach the limit to avoid large > ``` > Typo: reach -> reaches > Fixed -- Matheus Alcantara EDB: https://www.enterprisedb.com From f4dcd9d836137589c8345b74cb24ab3e7dc18eeb Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Wed, 26 Nov 2025 16:34:46 -0300 Subject: [PATCH v9] postgres_fdw: speed up batch inserts using COPY This commit include a new foreign table/server option "batch_with_copy_threshold" that enable the usage of the COPY command to speed up batch inserts when a COPY FROM or an insert into a table partition that is a foreign table is executed. In both cases the BeginForeignInsert fdw routine is called, so this new option is retrieved only on this routine. For the other cases that use the ForeignModify routines still use the INSERT as a remote SQL. Note that the COPY will only be used for batch inserts and only if the current number of rows being inserted on the batch operation is >= batch_with_copy_threshold. If batch_size=100, batch_with_copy_threshold=50 and number of rows being inserted is 120 the first 100 rows will be inserted using the COPY command and the remaining 20 rows will be inserted using INSERT statement because it did not reach the copy threshold. --- contrib/postgres_fdw/deparse.c | 35 +++ .../postgres_fdw/expected/postgres_fdw.out | 26 +++ contrib/postgres_fdw/option.c | 6 +- contrib/postgres_fdw/postgres_fdw.c | 215 +++++++++++++++++- contrib/postgres_fdw/postgres_fdw.h | 1 + contrib/postgres_fdw/sql/postgres_fdw.sql | 23 ++ 6 files changed, 303 insertions(+), 3 deletions(-) diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index ebe2c3a596a..78335db1889 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -2236,6 +2236,41 @@ rebuildInsertSql(StringInfo buf, Relation rel, appendStringInfoString(buf, orig_query + values_end_len); } +/* + * Build a COPY FROM STDIN statement using the TEXT format + */ +void +deparseCopySql(StringInfo buf, Relation rel, List *target_attrs) +{ + TupleDesc tupdesc = RelationGetDescr(rel); + bool first = true; + int nattrs = list_length(target_attrs); + + appendStringInfo(buf, "COPY "); + deparseRelation(buf, rel); + if (nattrs > 0) + appendStringInfo(buf, "("); + + foreach_int(attnum, target_attrs) + { + Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1); + + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoString(buf, ", "); + + first = false; + + appendStringInfoString(buf, quote_identifier(NameStr(attr->attname))); + } + if (nattrs > 0) + appendStringInfoString(buf, ") FROM STDIN"); + else + appendStringInfoString(buf, " FROM STDIN"); +} + /* * deparse remote UPDATE statement * diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index 6066510c7c0..8bbc27b7b3b 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -9215,6 +9215,19 @@ with result as (insert into itrtest values (1, 'test1'), (2, 'test2') returning drop trigger loct1_br_insert_trigger on loct1; drop trigger loct2_br_insert_trigger on loct2; +-- Test batch insert using COPY with batch_with_copy_threshold +delete from itrtest; +alter server loopback options (add batch_with_copy_threshold '2', batch_size '3'); +insert into itrtest values (1, 'test1'), (2, 'test2'), (2, 'test3'); +select * from itrtest; + a | b +---+------- + 1 | test1 + 2 | test2 + 2 | test3 +(3 rows) + +alter server loopback options (drop batch_with_copy_threshold, drop batch_size); drop table itrtest; drop table loct1; drop table loct2; @@ -9524,6 +9537,19 @@ select * from rem2; 2 | bar (2 rows) +delete from rem2; +-- Test COPY with batch_with_copy_threshold +alter foreign table rem2 options (add batch_with_copy_threshold '2'); +-- Insert 3 rows so that the third row fallback to normal INSERT statement path +copy rem2 from stdin; +select * from rem2; + f1 | f2 +----+----- + 1 | foo + 2 | bar + 3 | baz +(3 rows) + delete from rem2; -- Test check constraints alter table loc2 add constraint loc2_f1positive check (f1 >= 0); diff --git a/contrib/postgres_fdw/option.c b/contrib/postgres_fdw/option.c index b0bd72d1e58..4545c2f9ba1 100644 --- a/contrib/postgres_fdw/option.c +++ b/contrib/postgres_fdw/option.c @@ -157,7 +157,8 @@ postgres_fdw_validator(PG_FUNCTION_ARGS) (void) ExtractExtensionList(defGetString(def), true); } else if (strcmp(def->defname, "fetch_size") == 0 || - strcmp(def->defname, "batch_size") == 0) + strcmp(def->defname, "batch_size") == 0 || + strcmp(def->defname, "batch_with_copy_threshold") == 0) { char *value; int int_val; @@ -263,6 +264,9 @@ InitPgFdwOptions(void) /* batch_size is available on both server and table */ {"batch_size", ForeignServerRelationId, false}, {"batch_size", ForeignTableRelationId, false}, + /* batch_with_copy_threshold is available on both server and table */ + {"batch_with_copy_threshold", ForeignServerRelationId, false}, + {"batch_with_copy_threshold", ForeignTableRelationId, false}, /* async_capable is available on both server and table */ {"async_capable", ForeignServerRelationId, false}, {"async_capable", ForeignTableRelationId, false}, diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index 3572689e33b..2fb95167a1c 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -63,6 +63,9 @@ PG_MODULE_MAGIC_EXT( /* If no remote estimates, assume a sort costs 20% extra */ #define DEFAULT_FDW_SORT_MULTIPLIER 1.2 +/* Buffer size to send COPY IN data*/ +#define COPYBUFSIZ 8192 + /* * Indexes of FDW-private information stored in fdw_private lists. * @@ -198,6 +201,10 @@ typedef struct PgFdwModifyState bool has_returning; /* is there a RETURNING clause? */ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */ + /* COPY usage stuff */ + int batch_with_copy_threshold; /* value of FDW option */ + char *cmd_copy; /* COPY statement */ + /* info about parameters for prepared statement */ AttrNumber ctidAttno; /* attnum of input resjunk ctid column */ int p_nums; /* number of parameters to transmit */ @@ -545,6 +552,10 @@ static void merge_fdw_options(PgFdwRelationInfo *fpinfo, const PgFdwRelationInfo *fpinfo_o, const PgFdwRelationInfo *fpinfo_i); static int get_batch_size_option(Relation rel); +static int get_batch_with_copy_threshold(Relation rel); +static TupleTableSlot **execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots); /* @@ -2013,8 +2024,30 @@ postgresExecForeignBatchInsert(EState *estate, */ if (fmstate->aux_fmstate) resultRelInfo->ri_FdwState = fmstate->aux_fmstate; - rslot = execute_foreign_modify(estate, resultRelInfo, CMD_INSERT, - slots, planSlots, numSlots); + + /* + * Check if "batch_with_copy_threshold" is enable (> 0) and if the COPY + * can be used based on the number of rows being inserted on this batch. + * The original query also should not have a RETURNING clause. + */ + if (fmstate->batch_with_copy_threshold > 0 && + fmstate->batch_with_copy_threshold <= *numSlots && + !fmstate->has_returning) + { + if (fmstate->cmd_copy == NULL) + { + StringInfoData sql; + + initStringInfo(&sql); + deparseCopySql(&sql, fmstate->rel, fmstate->target_attrs); + fmstate->cmd_copy = sql.data; + } + + rslot = execute_foreign_modify_using_copy(fmstate, slots, numSlots); + } + else + rslot = execute_foreign_modify(estate, resultRelInfo, CMD_INSERT, + slots, planSlots, numSlots); /* Revert that change */ if (fmstate->aux_fmstate) resultRelInfo->ri_FdwState = fmstate; @@ -2265,6 +2298,16 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, retrieved_attrs != NIL, retrieved_attrs); + + /* + * Set batch_with_copy_threshold from foreign server/table options. We do + * this outside of create_foreign_modify() because we only want to use + * COPY as a remote SQL when a COPY FROM on a foreign table is executed or + * an insert is being performed on a table partition. In both cases the + * BeginForeignInsert fdw routine is called. + */ + fmstate->batch_with_copy_threshold = get_batch_with_copy_threshold(rel); + /* * If the given resultRelInfo already has PgFdwModifyState set, it means * the foreign table is an UPDATE subplan result rel; in which case, store @@ -4066,6 +4109,50 @@ create_foreign_modify(EState *estate, return fmstate; } +/* + * Write target attribute values from fmstate into buf buffer to be sent as + * COPY FROM STDIN data + */ +static void +convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot) +{ + TupleDesc tupdesc = RelationGetDescr(fmstate->rel); + bool first = true; + int i = 0; + + foreach_int(attnum, fmstate->target_attrs) + { + CompactAttribute *attr = TupleDescCompactAttr(tupdesc, attnum - 1); + Datum datum; + bool isnull; + + /* Ignore generated columns; they are set to DEFAULT */ + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoCharMacro(buf, '\t'); + first = false; + + datum = slot_getattr(slot, attnum, &isnull); + + if (isnull) + appendStringInfoString(buf, "\\N"); + else + { + const char *value = OutputFunctionCall(&fmstate->p_flinfo[i], + datum); + + appendStringInfoString(buf, value); + } + i++; + } + + appendStringInfoCharMacro(buf, '\n'); +} + /* * execute_foreign_modify * Perform foreign-table modification as required, and fetch RETURNING @@ -7886,3 +7973,127 @@ get_batch_size_option(Relation rel) return batch_size; } + +/* + * Determine COPY usage threshold for batching inserts for a given foreign + * table. The option specified for a table has precedence. + */ +static int +get_batch_with_copy_threshold(Relation rel) +{ + Oid foreigntableid = RelationGetRelid(rel); + List *options = NIL; + ListCell *lc; + ForeignTable *table; + ForeignServer *server; + + /* + * We use 0 as default, which means that COPY will not be used by default + * for batching insert. + */ + int copy_for_batch_insert_threshold = 0; + + /* + * Load options for table and server. We append server options after table + * options, because table options take precedence. + */ + table = GetForeignTable(foreigntableid); + server = GetForeignServer(table->serverid); + + options = list_concat(options, table->options); + options = list_concat(options, server->options); + + /* See if either table or server specifies enable_batch_with_copy. */ + foreach(lc, options) + { + DefElem *def = (DefElem *) lfirst(lc); + + if (strcmp(def->defname, "batch_with_copy_threshold") == 0) + { + (void) parse_int(defGetString(def), ©_for_batch_insert_threshold, 0, NULL); + break; + } + } + return copy_for_batch_insert_threshold; +} + +/* + * execute_foreign_modify_using_copy + * Perform foreign-table modification using the COPY command. + */ +static TupleTableSlot ** +execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots) +{ + PGresult *res; + StringInfoData copy_data; + int n_rows; + int i; + + Assert(fmstate->cmd_copy != NULL); + + /* Send COPY command */ + if (!PQsendQuery(fmstate->conn, fmstate->cmd_copy)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->cmd_copy); + + /* get the COPY result */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COPY_IN) + pgfdw_report_error(res, fmstate->conn, fmstate->cmd_copy); + + /* Clean up the COPY command result */ + PQclear(res); + + /* Convert the TupleTableSlot data into a TEXT-formatted line */ + initStringInfo(©_data); + for (i = 0; i < *numSlots; i++) + { + convert_slot_to_copy_text(©_data, fmstate, slots[i]); + + /* + * Send initial COPY data if the buffer reaches the limit to avoid + * large memory usage. + */ + if (copy_data.len >= COPYBUFSIZ) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->cmd_copy); + resetStringInfo(©_data); + } + } + + /* Send the remaining COPY data */ + if (copy_data.len > 0) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->cmd_copy); + } + + pfree(copy_data.data); + + /* End the COPY operation */ + if (PQputCopyEnd(fmstate->conn, NULL) < 0 || PQflush(fmstate->conn)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->cmd_copy); + + /* + * Get the result, and check for success. + */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + pgfdw_report_error(res, fmstate->conn, fmstate->cmd_copy); + + n_rows = atoi(PQcmdTuples(res)); + + /* And clean up */ + PQclear(res); + + MemoryContextReset(fmstate->temp_cxt); + + *numSlots = n_rows; + + /* + * Return NULL if nothing was inserted on the remote end + */ + return (n_rows > 0) ? slots : NULL; +} diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h index a2bb1ff352c..fc6922ddd4f 100644 --- a/contrib/postgres_fdw/postgres_fdw.h +++ b/contrib/postgres_fdw/postgres_fdw.h @@ -204,6 +204,7 @@ extern void rebuildInsertSql(StringInfo buf, Relation rel, char *orig_query, List *target_attrs, int values_end_len, int num_params, int num_rows); +extern void deparseCopySql(StringInfo buf, Relation rel, List *target_attrs); extern void deparseUpdateSql(StringInfo buf, RangeTblEntry *rte, Index rtindex, Relation rel, List *targetAttrs, diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql index 4f7ab2ed0ac..840e97fed2f 100644 --- a/contrib/postgres_fdw/sql/postgres_fdw.sql +++ b/contrib/postgres_fdw/sql/postgres_fdw.sql @@ -2643,6 +2643,16 @@ with result as (insert into itrtest values (1, 'test1'), (2, 'test2') returning drop trigger loct1_br_insert_trigger on loct1; drop trigger loct2_br_insert_trigger on loct2; +-- Test batch insert using COPY with batch_with_copy_threshold +delete from itrtest; +alter server loopback options (add batch_with_copy_threshold '2', batch_size '3'); + +insert into itrtest values (1, 'test1'), (2, 'test2'), (2, 'test3'); + +select * from itrtest; + +alter server loopback options (drop batch_with_copy_threshold, drop batch_size); + drop table itrtest; drop table loct1; drop table loct2; @@ -2815,6 +2825,19 @@ select * from rem2; delete from rem2; +-- Test COPY with batch_with_copy_threshold +alter foreign table rem2 options (add batch_with_copy_threshold '2'); + +-- Insert 3 rows so that the third row fallback to normal INSERT statement path +copy rem2 from stdin; +1 foo +2 bar +3 baz +\. +select * from rem2; + +delete from rem2; + -- Test check constraints alter table loc2 add constraint loc2_f1positive check (f1 >= 0); alter foreign table rem2 add constraint rem2_f1positive check (f1 >= 0); -- 2.51.2 Attachments: [text/plain] v9-0001-postgres_fdw-speed-up-batch-inserts-using-COPY.patch (14.8K, ../../[email protected]/2-v9-0001-postgres_fdw-speed-up-batch-inserts-using-COPY.patch) download | inline diff: From f4dcd9d836137589c8345b74cb24ab3e7dc18eeb Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Wed, 26 Nov 2025 16:34:46 -0300 Subject: [PATCH v9] postgres_fdw: speed up batch inserts using COPY This commit include a new foreign table/server option "batch_with_copy_threshold" that enable the usage of the COPY command to speed up batch inserts when a COPY FROM or an insert into a table partition that is a foreign table is executed. In both cases the BeginForeignInsert fdw routine is called, so this new option is retrieved only on this routine. For the other cases that use the ForeignModify routines still use the INSERT as a remote SQL. Note that the COPY will only be used for batch inserts and only if the current number of rows being inserted on the batch operation is >= batch_with_copy_threshold. If batch_size=100, batch_with_copy_threshold=50 and number of rows being inserted is 120 the first 100 rows will be inserted using the COPY command and the remaining 20 rows will be inserted using INSERT statement because it did not reach the copy threshold. --- contrib/postgres_fdw/deparse.c | 35 +++ .../postgres_fdw/expected/postgres_fdw.out | 26 +++ contrib/postgres_fdw/option.c | 6 +- contrib/postgres_fdw/postgres_fdw.c | 215 +++++++++++++++++- contrib/postgres_fdw/postgres_fdw.h | 1 + contrib/postgres_fdw/sql/postgres_fdw.sql | 23 ++ 6 files changed, 303 insertions(+), 3 deletions(-) diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index ebe2c3a596a..78335db1889 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -2236,6 +2236,41 @@ rebuildInsertSql(StringInfo buf, Relation rel, appendStringInfoString(buf, orig_query + values_end_len); } +/* + * Build a COPY FROM STDIN statement using the TEXT format + */ +void +deparseCopySql(StringInfo buf, Relation rel, List *target_attrs) +{ + TupleDesc tupdesc = RelationGetDescr(rel); + bool first = true; + int nattrs = list_length(target_attrs); + + appendStringInfo(buf, "COPY "); + deparseRelation(buf, rel); + if (nattrs > 0) + appendStringInfo(buf, "("); + + foreach_int(attnum, target_attrs) + { + Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1); + + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoString(buf, ", "); + + first = false; + + appendStringInfoString(buf, quote_identifier(NameStr(attr->attname))); + } + if (nattrs > 0) + appendStringInfoString(buf, ") FROM STDIN"); + else + appendStringInfoString(buf, " FROM STDIN"); +} + /* * deparse remote UPDATE statement * diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index 6066510c7c0..8bbc27b7b3b 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -9215,6 +9215,19 @@ with result as (insert into itrtest values (1, 'test1'), (2, 'test2') returning drop trigger loct1_br_insert_trigger on loct1; drop trigger loct2_br_insert_trigger on loct2; +-- Test batch insert using COPY with batch_with_copy_threshold +delete from itrtest; +alter server loopback options (add batch_with_copy_threshold '2', batch_size '3'); +insert into itrtest values (1, 'test1'), (2, 'test2'), (2, 'test3'); +select * from itrtest; + a | b +---+------- + 1 | test1 + 2 | test2 + 2 | test3 +(3 rows) + +alter server loopback options (drop batch_with_copy_threshold, drop batch_size); drop table itrtest; drop table loct1; drop table loct2; @@ -9524,6 +9537,19 @@ select * from rem2; 2 | bar (2 rows) +delete from rem2; +-- Test COPY with batch_with_copy_threshold +alter foreign table rem2 options (add batch_with_copy_threshold '2'); +-- Insert 3 rows so that the third row fallback to normal INSERT statement path +copy rem2 from stdin; +select * from rem2; + f1 | f2 +----+----- + 1 | foo + 2 | bar + 3 | baz +(3 rows) + delete from rem2; -- Test check constraints alter table loc2 add constraint loc2_f1positive check (f1 >= 0); diff --git a/contrib/postgres_fdw/option.c b/contrib/postgres_fdw/option.c index b0bd72d1e58..4545c2f9ba1 100644 --- a/contrib/postgres_fdw/option.c +++ b/contrib/postgres_fdw/option.c @@ -157,7 +157,8 @@ postgres_fdw_validator(PG_FUNCTION_ARGS) (void) ExtractExtensionList(defGetString(def), true); } else if (strcmp(def->defname, "fetch_size") == 0 || - strcmp(def->defname, "batch_size") == 0) + strcmp(def->defname, "batch_size") == 0 || + strcmp(def->defname, "batch_with_copy_threshold") == 0) { char *value; int int_val; @@ -263,6 +264,9 @@ InitPgFdwOptions(void) /* batch_size is available on both server and table */ {"batch_size", ForeignServerRelationId, false}, {"batch_size", ForeignTableRelationId, false}, + /* batch_with_copy_threshold is available on both server and table */ + {"batch_with_copy_threshold", ForeignServerRelationId, false}, + {"batch_with_copy_threshold", ForeignTableRelationId, false}, /* async_capable is available on both server and table */ {"async_capable", ForeignServerRelationId, false}, {"async_capable", ForeignTableRelationId, false}, diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index 3572689e33b..2fb95167a1c 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -63,6 +63,9 @@ PG_MODULE_MAGIC_EXT( /* If no remote estimates, assume a sort costs 20% extra */ #define DEFAULT_FDW_SORT_MULTIPLIER 1.2 +/* Buffer size to send COPY IN data*/ +#define COPYBUFSIZ 8192 + /* * Indexes of FDW-private information stored in fdw_private lists. * @@ -198,6 +201,10 @@ typedef struct PgFdwModifyState bool has_returning; /* is there a RETURNING clause? */ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */ + /* COPY usage stuff */ + int batch_with_copy_threshold; /* value of FDW option */ + char *cmd_copy; /* COPY statement */ + /* info about parameters for prepared statement */ AttrNumber ctidAttno; /* attnum of input resjunk ctid column */ int p_nums; /* number of parameters to transmit */ @@ -545,6 +552,10 @@ static void merge_fdw_options(PgFdwRelationInfo *fpinfo, const PgFdwRelationInfo *fpinfo_o, const PgFdwRelationInfo *fpinfo_i); static int get_batch_size_option(Relation rel); +static int get_batch_with_copy_threshold(Relation rel); +static TupleTableSlot **execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots); /* @@ -2013,8 +2024,30 @@ postgresExecForeignBatchInsert(EState *estate, */ if (fmstate->aux_fmstate) resultRelInfo->ri_FdwState = fmstate->aux_fmstate; - rslot = execute_foreign_modify(estate, resultRelInfo, CMD_INSERT, - slots, planSlots, numSlots); + + /* + * Check if "batch_with_copy_threshold" is enable (> 0) and if the COPY + * can be used based on the number of rows being inserted on this batch. + * The original query also should not have a RETURNING clause. + */ + if (fmstate->batch_with_copy_threshold > 0 && + fmstate->batch_with_copy_threshold <= *numSlots && + !fmstate->has_returning) + { + if (fmstate->cmd_copy == NULL) + { + StringInfoData sql; + + initStringInfo(&sql); + deparseCopySql(&sql, fmstate->rel, fmstate->target_attrs); + fmstate->cmd_copy = sql.data; + } + + rslot = execute_foreign_modify_using_copy(fmstate, slots, numSlots); + } + else + rslot = execute_foreign_modify(estate, resultRelInfo, CMD_INSERT, + slots, planSlots, numSlots); /* Revert that change */ if (fmstate->aux_fmstate) resultRelInfo->ri_FdwState = fmstate; @@ -2265,6 +2298,16 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, retrieved_attrs != NIL, retrieved_attrs); + + /* + * Set batch_with_copy_threshold from foreign server/table options. We do + * this outside of create_foreign_modify() because we only want to use + * COPY as a remote SQL when a COPY FROM on a foreign table is executed or + * an insert is being performed on a table partition. In both cases the + * BeginForeignInsert fdw routine is called. + */ + fmstate->batch_with_copy_threshold = get_batch_with_copy_threshold(rel); + /* * If the given resultRelInfo already has PgFdwModifyState set, it means * the foreign table is an UPDATE subplan result rel; in which case, store @@ -4066,6 +4109,50 @@ create_foreign_modify(EState *estate, return fmstate; } +/* + * Write target attribute values from fmstate into buf buffer to be sent as + * COPY FROM STDIN data + */ +static void +convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot) +{ + TupleDesc tupdesc = RelationGetDescr(fmstate->rel); + bool first = true; + int i = 0; + + foreach_int(attnum, fmstate->target_attrs) + { + CompactAttribute *attr = TupleDescCompactAttr(tupdesc, attnum - 1); + Datum datum; + bool isnull; + + /* Ignore generated columns; they are set to DEFAULT */ + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoCharMacro(buf, '\t'); + first = false; + + datum = slot_getattr(slot, attnum, &isnull); + + if (isnull) + appendStringInfoString(buf, "\\N"); + else + { + const char *value = OutputFunctionCall(&fmstate->p_flinfo[i], + datum); + + appendStringInfoString(buf, value); + } + i++; + } + + appendStringInfoCharMacro(buf, '\n'); +} + /* * execute_foreign_modify * Perform foreign-table modification as required, and fetch RETURNING @@ -7886,3 +7973,127 @@ get_batch_size_option(Relation rel) return batch_size; } + +/* + * Determine COPY usage threshold for batching inserts for a given foreign + * table. The option specified for a table has precedence. + */ +static int +get_batch_with_copy_threshold(Relation rel) +{ + Oid foreigntableid = RelationGetRelid(rel); + List *options = NIL; + ListCell *lc; + ForeignTable *table; + ForeignServer *server; + + /* + * We use 0 as default, which means that COPY will not be used by default + * for batching insert. + */ + int copy_for_batch_insert_threshold = 0; + + /* + * Load options for table and server. We append server options after table + * options, because table options take precedence. + */ + table = GetForeignTable(foreigntableid); + server = GetForeignServer(table->serverid); + + options = list_concat(options, table->options); + options = list_concat(options, server->options); + + /* See if either table or server specifies enable_batch_with_copy. */ + foreach(lc, options) + { + DefElem *def = (DefElem *) lfirst(lc); + + if (strcmp(def->defname, "batch_with_copy_threshold") == 0) + { + (void) parse_int(defGetString(def), ©_for_batch_insert_threshold, 0, NULL); + break; + } + } + return copy_for_batch_insert_threshold; +} + +/* + * execute_foreign_modify_using_copy + * Perform foreign-table modification using the COPY command. + */ +static TupleTableSlot ** +execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots) +{ + PGresult *res; + StringInfoData copy_data; + int n_rows; + int i; + + Assert(fmstate->cmd_copy != NULL); + + /* Send COPY command */ + if (!PQsendQuery(fmstate->conn, fmstate->cmd_copy)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->cmd_copy); + + /* get the COPY result */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COPY_IN) + pgfdw_report_error(res, fmstate->conn, fmstate->cmd_copy); + + /* Clean up the COPY command result */ + PQclear(res); + + /* Convert the TupleTableSlot data into a TEXT-formatted line */ + initStringInfo(©_data); + for (i = 0; i < *numSlots; i++) + { + convert_slot_to_copy_text(©_data, fmstate, slots[i]); + + /* + * Send initial COPY data if the buffer reaches the limit to avoid + * large memory usage. + */ + if (copy_data.len >= COPYBUFSIZ) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->cmd_copy); + resetStringInfo(©_data); + } + } + + /* Send the remaining COPY data */ + if (copy_data.len > 0) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->cmd_copy); + } + + pfree(copy_data.data); + + /* End the COPY operation */ + if (PQputCopyEnd(fmstate->conn, NULL) < 0 || PQflush(fmstate->conn)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->cmd_copy); + + /* + * Get the result, and check for success. + */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + pgfdw_report_error(res, fmstate->conn, fmstate->cmd_copy); + + n_rows = atoi(PQcmdTuples(res)); + + /* And clean up */ + PQclear(res); + + MemoryContextReset(fmstate->temp_cxt); + + *numSlots = n_rows; + + /* + * Return NULL if nothing was inserted on the remote end + */ + return (n_rows > 0) ? slots : NULL; +} diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h index a2bb1ff352c..fc6922ddd4f 100644 --- a/contrib/postgres_fdw/postgres_fdw.h +++ b/contrib/postgres_fdw/postgres_fdw.h @@ -204,6 +204,7 @@ extern void rebuildInsertSql(StringInfo buf, Relation rel, char *orig_query, List *target_attrs, int values_end_len, int num_params, int num_rows); +extern void deparseCopySql(StringInfo buf, Relation rel, List *target_attrs); extern void deparseUpdateSql(StringInfo buf, RangeTblEntry *rte, Index rtindex, Relation rel, List *targetAttrs, diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql index 4f7ab2ed0ac..840e97fed2f 100644 --- a/contrib/postgres_fdw/sql/postgres_fdw.sql +++ b/contrib/postgres_fdw/sql/postgres_fdw.sql @@ -2643,6 +2643,16 @@ with result as (insert into itrtest values (1, 'test1'), (2, 'test2') returning drop trigger loct1_br_insert_trigger on loct1; drop trigger loct2_br_insert_trigger on loct2; +-- Test batch insert using COPY with batch_with_copy_threshold +delete from itrtest; +alter server loopback options (add batch_with_copy_threshold '2', batch_size '3'); + +insert into itrtest values (1, 'test1'), (2, 'test2'), (2, 'test3'); + +select * from itrtest; + +alter server loopback options (drop batch_with_copy_threshold, drop batch_size); + drop table itrtest; drop table loct1; drop table loct2; @@ -2815,6 +2825,19 @@ select * from rem2; delete from rem2; +-- Test COPY with batch_with_copy_threshold +alter foreign table rem2 options (add batch_with_copy_threshold '2'); + +-- Insert 3 rows so that the third row fallback to normal INSERT statement path +copy rem2 from stdin; +1 foo +2 bar +3 baz +\. +select * from rem2; + +delete from rem2; + -- Test check constraints alter table loc2 add constraint loc2_f1positive check (f1 >= 0); alter foreign table rem2 add constraint rem2_f1positive check (f1 >= 0); -- 2.51.2 ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-01-27 19:17 Masahiko Sawada <[email protected]> parent: Matheus Alcantara <[email protected]> 1 sibling, 1 reply; 35+ messages in thread From: Masahiko Sawada @ 2026-01-27 19:17 UTC (permalink / raw) To: Matheus Alcantara <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers On Fri, Jan 2, 2026 at 12:33 PM Matheus Alcantara <[email protected]> wrote: > > On Fri Jan 2, 2026 at 5:15 PM -03, Masahiko Sawada wrote: > > + > > + /* > > + * Set batch_with_copy_threshold from foreign server/table options. We do > > + * this outside of create_foreign_modify() because we only want to use > > + * COPY as a remote SQL when a COPY FROM on a foreign table is executed or > > + * an insert is being performed on a table partition. In both cases the > > + * BeginForeignInsert fdw routine is called. > > + */ > > + fmstate->batch_with_copy_threshold = get_batch_with_copy_threshold(rel); > > > > Does it mean that we could end up using the COPY method not only when > > executing COPY FROM but also when executing INSERT with tuple > > routings? If so, how does the EXPLAIN command show the remote SQL? > > > It meas that we could also use the COPY method to insert rows into a > specific table partition that is a foreign table. > > Let's say that an user execute an INSERT INTO on a partitioned table > that has partitions that are postgres_fdw tables, with this patch we > could use the COPY method to insert the rows on these partitions. On > this scenario we would not have issue with EXPLAIN output because > currently we do not show the remote SQL being executed on each partition > that is involved on the INSERT statement. > > If an user execute an INSERT directly into a postgres_fdw table we will > use the normal INSERT statement as we use today. I'm slightly concerned that it could be confusing for users if we use the COPY method for the same table based on not only batch_with_copy_threshold but also how to INSERT. For example, if we insert tuples directly to a leaf partition, we always use INSERT. On the other hand, if we insert tuples via its parent table, we would use either COPY or INSERT based on the number of tuples and batch_with_copy_threshold value. IIUC this behavior stems from FDW API design (BeginForeignInsert callback is called only in cases of COPY or tuple routing), which users would not be aware of in general. Also, inserting tuples directly to a leaf partition is faster in general than doing via the parent table, but the COPY method optimization is available only in the latter case. How about making use of COPY method only when users execute a COPY command? Which seems more intuitive and a good start. We can distinguish BeginForeignInsert called via COPY from called via INSERT (tuple routing) by adding a flag to ModifyTableState or by checking if the passed resultRelInfo == resultRelInfo->ri_RootResultRelInfo. Alternative idea (or an improvement) would be to use the COPY method whenever the number of buffered tuples exceeds the threshold. It would cover more cases. Regarding the issue with EXPLAIN output, we could output both queries (INSERT and COPY) with some contexts (e.g., the threshold for the COPY method etc). Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-01-29 14:02 Matheus Alcantara <[email protected]> parent: Masahiko Sawada <[email protected]> 0 siblings, 1 reply; 35+ messages in thread From: Matheus Alcantara @ 2026-01-29 14:02 UTC (permalink / raw) To: Masahiko Sawada <[email protected]>; Matheus Alcantara <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers On Tue Jan 27, 2026 at 4:17 PM -03, Masahiko Sawada wrote: > On Fri, Jan 2, 2026 at 12:33 PM Matheus Alcantara > <[email protected]> wrote: >> >> On Fri Jan 2, 2026 at 5:15 PM -03, Masahiko Sawada wrote: >> > + >> > + /* >> > + * Set batch_with_copy_threshold from foreign server/table options. We do >> > + * this outside of create_foreign_modify() because we only want to use >> > + * COPY as a remote SQL when a COPY FROM on a foreign table is executed or >> > + * an insert is being performed on a table partition. In both cases the >> > + * BeginForeignInsert fdw routine is called. >> > + */ >> > + fmstate->batch_with_copy_threshold = get_batch_with_copy_threshold(rel); >> > >> > Does it mean that we could end up using the COPY method not only when >> > executing COPY FROM but also when executing INSERT with tuple >> > routings? If so, how does the EXPLAIN command show the remote SQL? >> > >> It meas that we could also use the COPY method to insert rows into a >> specific table partition that is a foreign table. >> >> Let's say that an user execute an INSERT INTO on a partitioned table >> that has partitions that are postgres_fdw tables, with this patch we >> could use the COPY method to insert the rows on these partitions. On >> this scenario we would not have issue with EXPLAIN output because >> currently we do not show the remote SQL being executed on each partition >> that is involved on the INSERT statement. >> >> If an user execute an INSERT directly into a postgres_fdw table we will >> use the normal INSERT statement as we use today. > > I'm slightly concerned that it could be confusing for users if we use > the COPY method for the same table based on not only > batch_with_copy_threshold but also how to INSERT. For example, if we > insert tuples directly to a leaf partition, we always use INSERT. On > the other hand, if we insert tuples via its parent table, we would use > either COPY or INSERT based on the number of tuples and > batch_with_copy_threshold value. IIUC this behavior stems from FDW API > design (BeginForeignInsert callback is called only in cases of COPY or > tuple routing), which users would not be aware of in general. Also, > inserting tuples directly to a leaf partition is faster in general > than doing via the parent table, but the COPY method optimization is > available only in the latter case. Yeah, I agree that this patch ends up in a land that it could introduce more confusing than improvements for the user. > How about making use of COPY method only when users execute a COPY > command? Which seems more intuitive and a good start. We can > distinguish BeginForeignInsert called via COPY from called via INSERT > (tuple routing) by adding a flag to ModifyTableState or by checking if > the passed resultRelInfo == resultRelInfo->ri_RootResultRelInfo. This sounds a good idea, it simplify the patch scope a lot. During my tests I've noticed that ri_RootResultRelInfo is null when it's being called by CopyFrom(), so on postgresBeginForeignInsert I've included a check that if it's null it means that it's being executed by a COPY command and then we could use the COPY command as remote SQL. Note that using COPY as the remote SQL is not always feasible. If the remote table has a trigger that modifies the row, and the local foreign table also has an insert trigger, we cannot capture those changes. While postgres_fdw typically relies on INSERT ... RETURNING * to synchronize the TupleTableSlot with remote side effects, the COPY command does not support a RETURNING clause. Without this synchronization, local triggers would see the original data rather than the actual values inserted. This limitation is why the ri_TrigDesc == NULL check is necessary; removing it causes the "Test a combination of local and remote triggers" regression test on postgres_fdw.sql to fail. > Alternative idea (or an improvement) would be to use the COPY method > whenever the number of buffered tuples exceeds the threshold. It would > cover more cases. Regarding the issue with EXPLAIN output, we could > output both queries (INSERT and COPY) with some contexts (e.g., the > threshold for the COPY method etc). We could implement this as a future improvement. -- Matheus Alcantara EDB: https://www.enterprisedb.com From 54f1194ef096b74ecf2405cc6afff95902885189 Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Wed, 28 Jan 2026 19:55:48 -0300 Subject: [PATCH v10] postgres_fdw: Use COPY as remote SQL when possible Previously when an user execute a COPY on a foreign table, postgres_fdw send a INSERT as a remote SQL to the foreign server. This commit introduce the ability to use the COPY command instead. The COPY command will only be used when an user execute a COPY on a foreign table and also the foreign table should not have triggers because remote triggers might modify the inserted row and since COPY does not support a RETURNING clause, we cannot synchronize the local TupleTableSlot with those changes for use in local AFTER triggers, so if the foreign table has any trigger INSERT will be used. Discussion: https://www.postgresql.org/message-id/flat/DDIZJ217OUDK.2R5WE4OGL5PTY%40gmail.com --- contrib/postgres_fdw/deparse.c | 36 ++++ .../postgres_fdw/expected/postgres_fdw.out | 6 +- contrib/postgres_fdw/postgres_fdw.c | 169 +++++++++++++++++- contrib/postgres_fdw/postgres_fdw.h | 1 + 4 files changed, 206 insertions(+), 6 deletions(-) diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index ebe2c3a596a..04829b6ee45 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -2177,6 +2177,42 @@ deparseInsertSql(StringInfo buf, RangeTblEntry *rte, withCheckOptionList, returningList, retrieved_attrs); } +/* + * Build a COPY FROM STDIN statement using the TEXT format + */ +void +deparseCopySql(StringInfo buf, Relation rel, List *target_attrs) +{ + TupleDesc tupdesc = RelationGetDescr(rel); + bool first = true; + int nattrs = list_length(target_attrs); + + appendStringInfo(buf, "COPY "); + deparseRelation(buf, rel); + if (nattrs > 0) + appendStringInfo(buf, "("); + + foreach_int(attnum, target_attrs) + { + Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1); + + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoString(buf, ", "); + + first = false; + + appendStringInfoString(buf, quote_identifier(NameStr(attr->attname))); + } + if (nattrs > 0) + appendStringInfoString(buf, ") FROM STDIN"); + else + appendStringInfoString(buf, " FROM STDIN"); +} + + /* * rebuild remote INSERT statement * diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index 6066510c7c0..2725f067223 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -9533,7 +9533,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN COPY rem2, line 1: "-1 xyzzy" select * from rem2; f1 | f2 @@ -9690,7 +9691,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN COPY rem2 select * from rem2; f1 | f2 diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index 3572689e33b..9630bae3146 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -63,6 +63,9 @@ PG_MODULE_MAGIC_EXT( /* If no remote estimates, assume a sort costs 20% extra */ #define DEFAULT_FDW_SORT_MULTIPLIER 1.2 +/* Buffer size to send COPY IN data*/ +#define COPYBUFSIZ 8192 + /* * Indexes of FDW-private information stored in fdw_private lists. * @@ -198,6 +201,8 @@ typedef struct PgFdwModifyState bool has_returning; /* is there a RETURNING clause? */ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */ + bool use_copy; + /* info about parameters for prepared statement */ AttrNumber ctidAttno; /* attnum of input resjunk ctid column */ int p_nums; /* number of parameters to transmit */ @@ -545,6 +550,12 @@ static void merge_fdw_options(PgFdwRelationInfo *fpinfo, const PgFdwRelationInfo *fpinfo_o, const PgFdwRelationInfo *fpinfo_i); static int get_batch_size_option(Relation rel); +static TupleTableSlot **execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots); +static void convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot); /* @@ -2170,6 +2181,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, List *targetAttrs = NIL; List *retrieved_attrs = NIL; bool doNothing = false; + bool useCopy = false; /* * If the foreign table we are about to insert routed rows into is also an @@ -2247,11 +2259,31 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, rte = exec_rt_fetch(resultRelation, estate); } + /* + * We can use COPY for remote inserts only if the following conditions are + * met: 1. The insert is not part of a partitioned table's tuple routing + * (ri_RootResultRelInfo == NULL). 2. There are no local triggers on the + * foreign table (ri_TrigDesc == NULL). Remote triggers might modify the + * inserted row; since COPY does not support a RETURNING clause, we cannot + * synchronize the local TupleTableSlot with those changes for use in + * local AFTER triggers. 3. There is no explicit RETURNING clause + * requested by the query. + */ + useCopy = resultRelInfo->ri_RootResultRelInfo == NULL + && resultRelInfo->ri_TrigDesc == NULL + && resultRelInfo->ri_returningList == NIL; + /* Construct the SQL command string. */ - deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, - resultRelInfo->ri_WithCheckOptions, - resultRelInfo->ri_returningList, - &retrieved_attrs, &values_end_len); + if (useCopy) + { + deparseCopySql(&sql, rel, targetAttrs); + values_end_len = 0; /* Keep compiler quiet */ + } + else + deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, + resultRelInfo->ri_WithCheckOptions, + resultRelInfo->ri_returningList, + &retrieved_attrs, &values_end_len); /* Construct an execution state. */ fmstate = create_foreign_modify(mtstate->ps.state, @@ -2264,6 +2296,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, values_end_len, retrieved_attrs != NIL, retrieved_attrs); + fmstate->use_copy = useCopy; /* * If the given resultRelInfo already has PgFdwModifyState set, it means @@ -4093,6 +4126,9 @@ execute_foreign_modify(EState *estate, operation == CMD_UPDATE || operation == CMD_DELETE); + if (fmstate->use_copy) + return execute_foreign_modify_using_copy(fmstate, slots, numSlots); + /* First, process a pending asynchronous request, if any. */ if (fmstate->conn_state->pendingAreq) process_pending_request(fmstate->conn_state->pendingAreq); @@ -7886,3 +7922,128 @@ get_batch_size_option(Relation rel) return batch_size; } + +/* + * execute_foreign_modify_using_copy + * Perform foreign-table modification using the COPY command. + */ +static TupleTableSlot ** +execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots) +{ + PGresult *res; + StringInfoData copy_data; + int n_rows; + int i; + + Assert(fmstate->use_copy == true); + + /* Send COPY command */ + if (!PQsendQuery(fmstate->conn, fmstate->query)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* get the COPY result */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COPY_IN) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + /* Clean up the COPY command result */ + PQclear(res); + + /* Convert the TupleTableSlot data into a TEXT-formatted line */ + initStringInfo(©_data); + for (i = 0; i < *numSlots; i++) + { + convert_slot_to_copy_text(©_data, fmstate, slots[i]); + + /* + * Send initial COPY data if the buffer reaches the limit to avoid + * large memory usage. + */ + if (copy_data.len >= COPYBUFSIZ) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + resetStringInfo(©_data); + } + } + + /* Send the remaining COPY data */ + if (copy_data.len > 0) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + } + + pfree(copy_data.data); + + /* End the COPY operation */ + if (PQputCopyEnd(fmstate->conn, NULL) < 0 || PQflush(fmstate->conn)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* + * Get the result, and check for success. + */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + n_rows = atoi(PQcmdTuples(res)); + + /* And clean up */ + PQclear(res); + + MemoryContextReset(fmstate->temp_cxt); + + *numSlots = n_rows; + + /* + * Return NULL if nothing was inserted on the remote end + */ + return (n_rows > 0) ? slots : NULL; +} + +/* + * Write target attribute values from fmstate into buf buffer to be sent as + * COPY FROM STDIN data + */ +static void +convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot) +{ + TupleDesc tupdesc = RelationGetDescr(fmstate->rel); + bool first = true; + int i = 0; + + foreach_int(attnum, fmstate->target_attrs) + { + CompactAttribute *attr = TupleDescCompactAttr(tupdesc, attnum - 1); + Datum datum; + bool isnull; + + /* Ignore generated columns; they are set to DEFAULT */ + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoCharMacro(buf, '\t'); + first = false; + + datum = slot_getattr(slot, attnum, &isnull); + + if (isnull) + appendStringInfoString(buf, "\\N"); + else + { + const char *value = OutputFunctionCall(&fmstate->p_flinfo[i], + datum); + + appendStringInfoString(buf, value); + } + i++; + } + + appendStringInfoCharMacro(buf, '\n'); +} diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h index a2bb1ff352c..fc6922ddd4f 100644 --- a/contrib/postgres_fdw/postgres_fdw.h +++ b/contrib/postgres_fdw/postgres_fdw.h @@ -204,6 +204,7 @@ extern void rebuildInsertSql(StringInfo buf, Relation rel, char *orig_query, List *target_attrs, int values_end_len, int num_params, int num_rows); +extern void deparseCopySql(StringInfo buf, Relation rel, List *target_attrs); extern void deparseUpdateSql(StringInfo buf, RangeTblEntry *rte, Index rtindex, Relation rel, List *targetAttrs, -- 2.51.2 Attachments: [text/plain] v10-0001-postgres_fdw-Use-COPY-as-remote-SQL-when-possibl.patch (10.9K, ../../[email protected]/2-v10-0001-postgres_fdw-Use-COPY-as-remote-SQL-when-possibl.patch) download | inline diff: From 54f1194ef096b74ecf2405cc6afff95902885189 Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Wed, 28 Jan 2026 19:55:48 -0300 Subject: [PATCH v10] postgres_fdw: Use COPY as remote SQL when possible Previously when an user execute a COPY on a foreign table, postgres_fdw send a INSERT as a remote SQL to the foreign server. This commit introduce the ability to use the COPY command instead. The COPY command will only be used when an user execute a COPY on a foreign table and also the foreign table should not have triggers because remote triggers might modify the inserted row and since COPY does not support a RETURNING clause, we cannot synchronize the local TupleTableSlot with those changes for use in local AFTER triggers, so if the foreign table has any trigger INSERT will be used. Discussion: https://www.postgresql.org/message-id/flat/DDIZJ217OUDK.2R5WE4OGL5PTY%40gmail.com --- contrib/postgres_fdw/deparse.c | 36 ++++ .../postgres_fdw/expected/postgres_fdw.out | 6 +- contrib/postgres_fdw/postgres_fdw.c | 169 +++++++++++++++++- contrib/postgres_fdw/postgres_fdw.h | 1 + 4 files changed, 206 insertions(+), 6 deletions(-) diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index ebe2c3a596a..04829b6ee45 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -2177,6 +2177,42 @@ deparseInsertSql(StringInfo buf, RangeTblEntry *rte, withCheckOptionList, returningList, retrieved_attrs); } +/* + * Build a COPY FROM STDIN statement using the TEXT format + */ +void +deparseCopySql(StringInfo buf, Relation rel, List *target_attrs) +{ + TupleDesc tupdesc = RelationGetDescr(rel); + bool first = true; + int nattrs = list_length(target_attrs); + + appendStringInfo(buf, "COPY "); + deparseRelation(buf, rel); + if (nattrs > 0) + appendStringInfo(buf, "("); + + foreach_int(attnum, target_attrs) + { + Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1); + + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoString(buf, ", "); + + first = false; + + appendStringInfoString(buf, quote_identifier(NameStr(attr->attname))); + } + if (nattrs > 0) + appendStringInfoString(buf, ") FROM STDIN"); + else + appendStringInfoString(buf, " FROM STDIN"); +} + + /* * rebuild remote INSERT statement * diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index 6066510c7c0..2725f067223 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -9533,7 +9533,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN COPY rem2, line 1: "-1 xyzzy" select * from rem2; f1 | f2 @@ -9690,7 +9691,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN COPY rem2 select * from rem2; f1 | f2 diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index 3572689e33b..9630bae3146 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -63,6 +63,9 @@ PG_MODULE_MAGIC_EXT( /* If no remote estimates, assume a sort costs 20% extra */ #define DEFAULT_FDW_SORT_MULTIPLIER 1.2 +/* Buffer size to send COPY IN data*/ +#define COPYBUFSIZ 8192 + /* * Indexes of FDW-private information stored in fdw_private lists. * @@ -198,6 +201,8 @@ typedef struct PgFdwModifyState bool has_returning; /* is there a RETURNING clause? */ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */ + bool use_copy; + /* info about parameters for prepared statement */ AttrNumber ctidAttno; /* attnum of input resjunk ctid column */ int p_nums; /* number of parameters to transmit */ @@ -545,6 +550,12 @@ static void merge_fdw_options(PgFdwRelationInfo *fpinfo, const PgFdwRelationInfo *fpinfo_o, const PgFdwRelationInfo *fpinfo_i); static int get_batch_size_option(Relation rel); +static TupleTableSlot **execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots); +static void convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot); /* @@ -2170,6 +2181,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, List *targetAttrs = NIL; List *retrieved_attrs = NIL; bool doNothing = false; + bool useCopy = false; /* * If the foreign table we are about to insert routed rows into is also an @@ -2247,11 +2259,31 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, rte = exec_rt_fetch(resultRelation, estate); } + /* + * We can use COPY for remote inserts only if the following conditions are + * met: 1. The insert is not part of a partitioned table's tuple routing + * (ri_RootResultRelInfo == NULL). 2. There are no local triggers on the + * foreign table (ri_TrigDesc == NULL). Remote triggers might modify the + * inserted row; since COPY does not support a RETURNING clause, we cannot + * synchronize the local TupleTableSlot with those changes for use in + * local AFTER triggers. 3. There is no explicit RETURNING clause + * requested by the query. + */ + useCopy = resultRelInfo->ri_RootResultRelInfo == NULL + && resultRelInfo->ri_TrigDesc == NULL + && resultRelInfo->ri_returningList == NIL; + /* Construct the SQL command string. */ - deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, - resultRelInfo->ri_WithCheckOptions, - resultRelInfo->ri_returningList, - &retrieved_attrs, &values_end_len); + if (useCopy) + { + deparseCopySql(&sql, rel, targetAttrs); + values_end_len = 0; /* Keep compiler quiet */ + } + else + deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, + resultRelInfo->ri_WithCheckOptions, + resultRelInfo->ri_returningList, + &retrieved_attrs, &values_end_len); /* Construct an execution state. */ fmstate = create_foreign_modify(mtstate->ps.state, @@ -2264,6 +2296,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, values_end_len, retrieved_attrs != NIL, retrieved_attrs); + fmstate->use_copy = useCopy; /* * If the given resultRelInfo already has PgFdwModifyState set, it means @@ -4093,6 +4126,9 @@ execute_foreign_modify(EState *estate, operation == CMD_UPDATE || operation == CMD_DELETE); + if (fmstate->use_copy) + return execute_foreign_modify_using_copy(fmstate, slots, numSlots); + /* First, process a pending asynchronous request, if any. */ if (fmstate->conn_state->pendingAreq) process_pending_request(fmstate->conn_state->pendingAreq); @@ -7886,3 +7922,128 @@ get_batch_size_option(Relation rel) return batch_size; } + +/* + * execute_foreign_modify_using_copy + * Perform foreign-table modification using the COPY command. + */ +static TupleTableSlot ** +execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots) +{ + PGresult *res; + StringInfoData copy_data; + int n_rows; + int i; + + Assert(fmstate->use_copy == true); + + /* Send COPY command */ + if (!PQsendQuery(fmstate->conn, fmstate->query)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* get the COPY result */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COPY_IN) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + /* Clean up the COPY command result */ + PQclear(res); + + /* Convert the TupleTableSlot data into a TEXT-formatted line */ + initStringInfo(©_data); + for (i = 0; i < *numSlots; i++) + { + convert_slot_to_copy_text(©_data, fmstate, slots[i]); + + /* + * Send initial COPY data if the buffer reaches the limit to avoid + * large memory usage. + */ + if (copy_data.len >= COPYBUFSIZ) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + resetStringInfo(©_data); + } + } + + /* Send the remaining COPY data */ + if (copy_data.len > 0) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + } + + pfree(copy_data.data); + + /* End the COPY operation */ + if (PQputCopyEnd(fmstate->conn, NULL) < 0 || PQflush(fmstate->conn)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* + * Get the result, and check for success. + */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + n_rows = atoi(PQcmdTuples(res)); + + /* And clean up */ + PQclear(res); + + MemoryContextReset(fmstate->temp_cxt); + + *numSlots = n_rows; + + /* + * Return NULL if nothing was inserted on the remote end + */ + return (n_rows > 0) ? slots : NULL; +} + +/* + * Write target attribute values from fmstate into buf buffer to be sent as + * COPY FROM STDIN data + */ +static void +convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot) +{ + TupleDesc tupdesc = RelationGetDescr(fmstate->rel); + bool first = true; + int i = 0; + + foreach_int(attnum, fmstate->target_attrs) + { + CompactAttribute *attr = TupleDescCompactAttr(tupdesc, attnum - 1); + Datum datum; + bool isnull; + + /* Ignore generated columns; they are set to DEFAULT */ + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoCharMacro(buf, '\t'); + first = false; + + datum = slot_getattr(slot, attnum, &isnull); + + if (isnull) + appendStringInfoString(buf, "\\N"); + else + { + const char *value = OutputFunctionCall(&fmstate->p_flinfo[i], + datum); + + appendStringInfoString(buf, value); + } + i++; + } + + appendStringInfoCharMacro(buf, '\n'); +} diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h index a2bb1ff352c..fc6922ddd4f 100644 --- a/contrib/postgres_fdw/postgres_fdw.h +++ b/contrib/postgres_fdw/postgres_fdw.h @@ -204,6 +204,7 @@ extern void rebuildInsertSql(StringInfo buf, Relation rel, char *orig_query, List *target_attrs, int values_end_len, int num_params, int num_rows); +extern void deparseCopySql(StringInfo buf, Relation rel, List *target_attrs); extern void deparseUpdateSql(StringInfo buf, RangeTblEntry *rte, Index rtindex, Relation rel, List *targetAttrs, -- 2.51.2 ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-02-26 01:39 Masahiko Sawada <[email protected]> parent: Matheus Alcantara <[email protected]> 0 siblings, 1 reply; 35+ messages in thread From: Masahiko Sawada @ 2026-02-26 01:39 UTC (permalink / raw) To: Matheus Alcantara <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers On Thu, Jan 29, 2026 at 6:02 AM Matheus Alcantara <[email protected]> wrote: > > On Tue Jan 27, 2026 at 4:17 PM -03, Masahiko Sawada wrote: > > On Fri, Jan 2, 2026 at 12:33 PM Matheus Alcantara > > <[email protected]> wrote: > >> > >> On Fri Jan 2, 2026 at 5:15 PM -03, Masahiko Sawada wrote: > >> > + > >> > + /* > >> > + * Set batch_with_copy_threshold from foreign server/table options. We do > >> > + * this outside of create_foreign_modify() because we only want to use > >> > + * COPY as a remote SQL when a COPY FROM on a foreign table is executed or > >> > + * an insert is being performed on a table partition. In both cases the > >> > + * BeginForeignInsert fdw routine is called. > >> > + */ > >> > + fmstate->batch_with_copy_threshold = get_batch_with_copy_threshold(rel); > >> > > >> > Does it mean that we could end up using the COPY method not only when > >> > executing COPY FROM but also when executing INSERT with tuple > >> > routings? If so, how does the EXPLAIN command show the remote SQL? > >> > > >> It meas that we could also use the COPY method to insert rows into a > >> specific table partition that is a foreign table. > >> > >> Let's say that an user execute an INSERT INTO on a partitioned table > >> that has partitions that are postgres_fdw tables, with this patch we > >> could use the COPY method to insert the rows on these partitions. On > >> this scenario we would not have issue with EXPLAIN output because > >> currently we do not show the remote SQL being executed on each partition > >> that is involved on the INSERT statement. > >> > >> If an user execute an INSERT directly into a postgres_fdw table we will > >> use the normal INSERT statement as we use today. > > > > I'm slightly concerned that it could be confusing for users if we use > > the COPY method for the same table based on not only > > batch_with_copy_threshold but also how to INSERT. For example, if we > > insert tuples directly to a leaf partition, we always use INSERT. On > > the other hand, if we insert tuples via its parent table, we would use > > either COPY or INSERT based on the number of tuples and > > batch_with_copy_threshold value. IIUC this behavior stems from FDW API > > design (BeginForeignInsert callback is called only in cases of COPY or > > tuple routing), which users would not be aware of in general. Also, > > inserting tuples directly to a leaf partition is faster in general > > than doing via the parent table, but the COPY method optimization is > > available only in the latter case. > > Yeah, I agree that this patch ends up in a land that it could introduce > more confusing than improvements for the user. > > > How about making use of COPY method only when users execute a COPY > > command? Which seems more intuitive and a good start. We can > > distinguish BeginForeignInsert called via COPY from called via INSERT > > (tuple routing) by adding a flag to ModifyTableState or by checking if > > the passed resultRelInfo == resultRelInfo->ri_RootResultRelInfo. > > This sounds a good idea, it simplify the patch scope a lot. During my > tests I've noticed that ri_RootResultRelInfo is null when it's being > called by CopyFrom(), so on postgresBeginForeignInsert I've included a > check that if it's null it means that it's being executed by a COPY > command and then we could use the COPY command as remote SQL. Thank you for updating the patch! > Note that using COPY as the remote SQL is not always feasible. If the > remote table has a trigger that modifies the row, and the local foreign > table also has an insert trigger, we cannot capture those changes. While > postgres_fdw typically relies on INSERT ... RETURNING * to synchronize > the TupleTableSlot with remote side effects, the COPY command does not > support a RETURNING clause. Without this synchronization, local triggers > would see the original data rather than the actual values inserted. This > limitation is why the ri_TrigDesc == NULL check is necessary; removing > it causes the "Test a combination of local and remote triggers" > regression test on postgres_fdw.sql to fail. Agreed. If this problem happens only when the local table has an AFTER INSERT trigger, can we check ri_TrigDesc->trig_insert_after_row too? Regarding the third condition, resultRelInfo->ri_returningList == NIL, can we make it an Assert() because checking resultRelInfo->RootResultRelInfo == NULL already checks if it's called via COPY? One thing it might be worth considering is to add some regression tests verifying that COPY commands are actually being used on the remote server in success cases. That way, we can be aware of changes even if we change the assumption in the future that RootResultRelInfo == NULL only when postgresBeginForeignInsert() is called via COPY. One idea is to define a trigger on the remote server that checks if the executed query is INSERT or COPY. For example, create function insert_or_copy() returns trigger as $$ declare query text; begin query := current_query(); if query ~* '^COPY' then raise notice 'COPY command'; elsif query ~* '^INSERT' then raise notice 'INSERT command'; end if; return new; end; $$ language plpgsql; Note that we need to set client_min_message to 'log' so that we can write the notice message raised via postgres_fdw. Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-02-26 15:58 Matheus Alcantara <[email protected]> parent: Masahiko Sawada <[email protected]> 0 siblings, 2 replies; 35+ messages in thread From: Matheus Alcantara @ 2026-02-26 15:58 UTC (permalink / raw) To: Masahiko Sawada <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers On Wed Feb 25, 2026 at 10:39 PM -03, Masahiko Sawada wrote: >> Note that using COPY as the remote SQL is not always feasible. If the >> remote table has a trigger that modifies the row, and the local foreign >> table also has an insert trigger, we cannot capture those changes. While >> postgres_fdw typically relies on INSERT ... RETURNING * to synchronize >> the TupleTableSlot with remote side effects, the COPY command does not >> support a RETURNING clause. Without this synchronization, local triggers >> would see the original data rather than the actual values inserted. This >> limitation is why the ri_TrigDesc == NULL check is necessary; removing >> it causes the "Test a combination of local and remote triggers" >> regression test on postgres_fdw.sql to fail. > > Agreed. If this problem happens only when the local table has an AFTER > INSERT trigger, can we check ri_TrigDesc->trig_insert_after_row too? > Yes, it's better to only fallback to insert mode when the table have a AFTER trigger. Fixed. > Regarding the third condition, resultRelInfo->ri_returningList == NIL, > can we make it an Assert() because checking > resultRelInfo->RootResultRelInfo == NULL already checks if it's called > via COPY? > Yes, souns better. Fixed. > One thing it might be worth considering is to add some regression > tests verifying that COPY commands are actually being used on the > remote server in success cases. That way, we can be aware of changes > even if we change the assumption in the future that RootResultRelInfo > == NULL only when postgresBeginForeignInsert() is called via COPY. One > idea is to define a trigger on the remote server that checks if the > executed query is INSERT or COPY. For example, > > create function insert_or_copy() returns trigger as $$ > declare query text; > begin > query := current_query(); > if query ~* '^COPY' then > raise notice 'COPY command'; > elsif query ~* '^INSERT' then > raise notice 'INSERT command'; > end if; > return new; > end; > $$ language plpgsql; > > Note that we need to set client_min_message to 'log' so that we can > write the notice message raised via postgres_fdw. > Good, thanks for this! I've added on this new version. Please see the new attached version. Thank you for reviewing this! -- Matheus Alcantara EDB: https://www.enterprisedb.com From f569b0da822e9ad0bef521b7f4f8532a45948577 Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Wed, 28 Jan 2026 19:55:48 -0300 Subject: [PATCH v11] postgres_fdw: Use COPY as remote SQL when possible Previously when an user execute a COPY on a foreign table, postgres_fdw send a INSERT as a remote SQL to the foreign server. This commit introduce the ability to use the COPY command instead. The COPY command will only be used when an user execute a COPY on a foreign table and also the foreign table should not have triggers because remote triggers might modify the inserted row and since COPY does not support a RETURNING clause, we cannot synchronize the local TupleTableSlot with those changes for use in local AFTER triggers, so if the foreign table has any trigger INSERT will be used. Author: Matheus Alcantara <[email protected]> Reviewed-By: Tomas Vondra <[email protected]> Reviewed-By: Jakub Wartak <[email protected]> Reviewed-By: jian he <[email protected]> Reviewed-By: Dewei Dai <[email protected]> Reviewed-By: Masahiko Sawada <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/DDIZJ217OUDK.2R5WE4OGL5PTY%40gmail.com --- contrib/postgres_fdw/deparse.c | 36 ++++ .../postgres_fdw/expected/postgres_fdw.out | 33 +++- contrib/postgres_fdw/postgres_fdw.c | 179 +++++++++++++++++- contrib/postgres_fdw/postgres_fdw.h | 1 + contrib/postgres_fdw/sql/postgres_fdw.sql | 27 +++ 5 files changed, 268 insertions(+), 8 deletions(-) diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index ebe2c3a596a..04829b6ee45 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -2177,6 +2177,42 @@ deparseInsertSql(StringInfo buf, RangeTblEntry *rte, withCheckOptionList, returningList, retrieved_attrs); } +/* + * Build a COPY FROM STDIN statement using the TEXT format + */ +void +deparseCopySql(StringInfo buf, Relation rel, List *target_attrs) +{ + TupleDesc tupdesc = RelationGetDescr(rel); + bool first = true; + int nattrs = list_length(target_attrs); + + appendStringInfo(buf, "COPY "); + deparseRelation(buf, rel); + if (nattrs > 0) + appendStringInfo(buf, "("); + + foreach_int(attnum, target_attrs) + { + Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1); + + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoString(buf, ", "); + + first = false; + + appendStringInfoString(buf, quote_identifier(NameStr(attr->attname))); + } + if (nattrs > 0) + appendStringInfoString(buf, ") FROM STDIN"); + else + appendStringInfoString(buf, " FROM STDIN"); +} + + /* * rebuild remote INSERT statement * diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index 2ccb72c539a..22c2dcdb6b1 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -7603,6 +7603,27 @@ select * from grem1; (2 rows) delete from grem1; +-- test that fdw also use COPY FROM as a remote sql +set client_min_messages to 'log'; +create function insert_or_copy() returns trigger as $$ +declare query text; +begin + query := current_query(); + if query ~* '^COPY' then + raise notice 'COPY command'; + elsif query ~* '^INSERT' then + raise notice 'INSERT command'; + end if; +return new; +end; +$$ language plpgsql; +CREATE TRIGGER trig_row_before +BEFORE INSERT OR UPDATE OR DELETE ON gloc1 +FOR EACH ROW EXECUTE PROCEDURE insert_or_copy(); +copy grem1 from stdin; +LOG: received message via remote connection: NOTICE: COPY command +drop trigger trig_row_before on gloc1; +reset client_min_messages; -- test batch insert alter server loopback options (add batch_size '10'); explain (verbose, costs off) @@ -7620,16 +7641,18 @@ insert into grem1 (a) values (1), (2); select * from gloc1; a | b | c ---+---+--- + 3 | 6 | 1 | 2 | 2 | 4 | -(2 rows) +(3 rows) select * from grem1; a | b | c ---+---+--- + 3 | 6 | 9 1 | 2 | 3 2 | 4 | 6 -(2 rows) +(3 rows) delete from grem1; -- batch insert with foreign partitions. @@ -9544,7 +9567,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN COPY rem2, line 1: "-1 xyzzy" select * from rem2; f1 | f2 @@ -9701,7 +9725,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN COPY rem2 select * from rem2; f1 | f2 diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index 60d90329a65..6dd22e4043d 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -63,6 +63,9 @@ PG_MODULE_MAGIC_EXT( /* If no remote estimates, assume a sort costs 20% extra */ #define DEFAULT_FDW_SORT_MULTIPLIER 1.2 +/* Buffer size to send COPY IN data*/ +#define COPYBUFSIZ 8192 + /* * Indexes of FDW-private information stored in fdw_private lists. * @@ -198,6 +201,8 @@ typedef struct PgFdwModifyState bool has_returning; /* is there a RETURNING clause? */ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */ + bool use_copy; + /* info about parameters for prepared statement */ AttrNumber ctidAttno; /* attnum of input resjunk ctid column */ int p_nums; /* number of parameters to transmit */ @@ -545,6 +550,12 @@ static void merge_fdw_options(PgFdwRelationInfo *fpinfo, const PgFdwRelationInfo *fpinfo_o, const PgFdwRelationInfo *fpinfo_i); static int get_batch_size_option(Relation rel); +static TupleTableSlot **execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots); +static void convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot); /* @@ -2170,6 +2181,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, List *targetAttrs = NIL; List *retrieved_attrs = NIL; bool doNothing = false; + bool useCopy = false; /* * If the foreign table we are about to insert routed rows into is also an @@ -2247,11 +2259,41 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, rte = exec_rt_fetch(resultRelation, estate); } + /* + * We can use COPY for remote inserts only if all the following conditions + * are met: + * + * Direct Execution: The command is a COPY FROM on the foreign table + * itself, not part of a partitioned table's tuple routing. ( + * resultRelInfo->ri_RootResultRelInfo == NULL) + * + * No Local AFTER Triggers: There are no AFTER ROW triggers defined locally + * on the foreign table. + * + * Remote triggers might modify the inserted row. Because the COPY protocol + * does not support a RETURNING clause, we cannot retrieve those changes to + * synchronize the local TupleTableSlot required by local AFTER triggers. + */ + if (resultRelInfo->ri_RootResultRelInfo == NULL) + { + /* There is no RETURNING clause on COPY */ + Assert(resultRelInfo->ri_returningList == NIL); + + useCopy = (resultRelInfo->ri_TrigDesc == NULL || + !resultRelInfo->ri_TrigDesc->trig_insert_after_row); + } + /* Construct the SQL command string. */ - deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, - resultRelInfo->ri_WithCheckOptions, - resultRelInfo->ri_returningList, - &retrieved_attrs, &values_end_len); + if (useCopy) + { + deparseCopySql(&sql, rel, targetAttrs); + values_end_len = 0; /* Keep compiler quiet */ + } + else + deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, + resultRelInfo->ri_WithCheckOptions, + resultRelInfo->ri_returningList, + &retrieved_attrs, &values_end_len); /* Construct an execution state. */ fmstate = create_foreign_modify(mtstate->ps.state, @@ -2264,6 +2306,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, values_end_len, retrieved_attrs != NIL, retrieved_attrs); + fmstate->use_copy = useCopy; /* * If the given resultRelInfo already has PgFdwModifyState set, it means @@ -4093,6 +4136,9 @@ execute_foreign_modify(EState *estate, operation == CMD_UPDATE || operation == CMD_DELETE); + if (fmstate->use_copy) + return execute_foreign_modify_using_copy(fmstate, slots, numSlots); + /* First, process a pending asynchronous request, if any. */ if (fmstate->conn_state->pendingAreq) process_pending_request(fmstate->conn_state->pendingAreq); @@ -7886,3 +7932,128 @@ get_batch_size_option(Relation rel) return batch_size; } + +/* + * execute_foreign_modify_using_copy + * Perform foreign-table modification using the COPY command. + */ +static TupleTableSlot ** +execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots) +{ + PGresult *res; + StringInfoData copy_data; + int n_rows; + int i; + + Assert(fmstate->use_copy == true); + + /* Send COPY command */ + if (!PQsendQuery(fmstate->conn, fmstate->query)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* get the COPY result */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COPY_IN) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + /* Clean up the COPY command result */ + PQclear(res); + + /* Convert the TupleTableSlot data into a TEXT-formatted line */ + initStringInfo(©_data); + for (i = 0; i < *numSlots; i++) + { + convert_slot_to_copy_text(©_data, fmstate, slots[i]); + + /* + * Send initial COPY data if the buffer reaches the limit to avoid + * large memory usage. + */ + if (copy_data.len >= COPYBUFSIZ) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + resetStringInfo(©_data); + } + } + + /* Send the remaining COPY data */ + if (copy_data.len > 0) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + } + + pfree(copy_data.data); + + /* End the COPY operation */ + if (PQputCopyEnd(fmstate->conn, NULL) < 0 || PQflush(fmstate->conn)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* + * Get the result, and check for success. + */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + n_rows = atoi(PQcmdTuples(res)); + + /* And clean up */ + PQclear(res); + + MemoryContextReset(fmstate->temp_cxt); + + *numSlots = n_rows; + + /* + * Return NULL if nothing was inserted on the remote end + */ + return (n_rows > 0) ? slots : NULL; +} + +/* + * Write target attribute values from fmstate into buf buffer to be sent as + * COPY FROM STDIN data + */ +static void +convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot) +{ + TupleDesc tupdesc = RelationGetDescr(fmstate->rel); + bool first = true; + int i = 0; + + foreach_int(attnum, fmstate->target_attrs) + { + CompactAttribute *attr = TupleDescCompactAttr(tupdesc, attnum - 1); + Datum datum; + bool isnull; + + /* Ignore generated columns; they are set to DEFAULT */ + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoCharMacro(buf, '\t'); + first = false; + + datum = slot_getattr(slot, attnum, &isnull); + + if (isnull) + appendStringInfoString(buf, "\\N"); + else + { + const char *value = OutputFunctionCall(&fmstate->p_flinfo[i], + datum); + + appendStringInfoString(buf, value); + } + i++; + } + + appendStringInfoCharMacro(buf, '\n'); +} diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h index a2bb1ff352c..fc6922ddd4f 100644 --- a/contrib/postgres_fdw/postgres_fdw.h +++ b/contrib/postgres_fdw/postgres_fdw.h @@ -204,6 +204,7 @@ extern void rebuildInsertSql(StringInfo buf, Relation rel, char *orig_query, List *target_attrs, int values_end_len, int num_params, int num_rows); +extern void deparseCopySql(StringInfo buf, Relation rel, List *target_attrs); extern void deparseUpdateSql(StringInfo buf, RangeTblEntry *rte, Index rtindex, Relation rel, List *targetAttrs, diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql index 72d2d9c311b..90246ddbd02 100644 --- a/contrib/postgres_fdw/sql/postgres_fdw.sql +++ b/contrib/postgres_fdw/sql/postgres_fdw.sql @@ -1929,6 +1929,33 @@ select * from gloc1; select * from grem1; delete from grem1; +-- test that fdw also use COPY FROM as a remote sql +set client_min_messages to 'log'; + +create function insert_or_copy() returns trigger as $$ +declare query text; +begin + query := current_query(); + if query ~* '^COPY' then + raise notice 'COPY command'; + elsif query ~* '^INSERT' then + raise notice 'INSERT command'; + end if; +return new; +end; +$$ language plpgsql; + +CREATE TRIGGER trig_row_before +BEFORE INSERT OR UPDATE OR DELETE ON gloc1 +FOR EACH ROW EXECUTE PROCEDURE insert_or_copy(); + +copy grem1 from stdin; +3 +\. + +drop trigger trig_row_before on gloc1; +reset client_min_messages; + -- test batch insert alter server loopback options (add batch_size '10'); explain (verbose, costs off) -- 2.52.0 Attachments: [text/plain] v11-0001-postgres_fdw-Use-COPY-as-remote-SQL-when-possibl.patch (13.7K, ../../[email protected]/2-v11-0001-postgres_fdw-Use-COPY-as-remote-SQL-when-possibl.patch) download | inline diff: From f569b0da822e9ad0bef521b7f4f8532a45948577 Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Wed, 28 Jan 2026 19:55:48 -0300 Subject: [PATCH v11] postgres_fdw: Use COPY as remote SQL when possible Previously when an user execute a COPY on a foreign table, postgres_fdw send a INSERT as a remote SQL to the foreign server. This commit introduce the ability to use the COPY command instead. The COPY command will only be used when an user execute a COPY on a foreign table and also the foreign table should not have triggers because remote triggers might modify the inserted row and since COPY does not support a RETURNING clause, we cannot synchronize the local TupleTableSlot with those changes for use in local AFTER triggers, so if the foreign table has any trigger INSERT will be used. Author: Matheus Alcantara <[email protected]> Reviewed-By: Tomas Vondra <[email protected]> Reviewed-By: Jakub Wartak <[email protected]> Reviewed-By: jian he <[email protected]> Reviewed-By: Dewei Dai <[email protected]> Reviewed-By: Masahiko Sawada <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/DDIZJ217OUDK.2R5WE4OGL5PTY%40gmail.com --- contrib/postgres_fdw/deparse.c | 36 ++++ .../postgres_fdw/expected/postgres_fdw.out | 33 +++- contrib/postgres_fdw/postgres_fdw.c | 179 +++++++++++++++++- contrib/postgres_fdw/postgres_fdw.h | 1 + contrib/postgres_fdw/sql/postgres_fdw.sql | 27 +++ 5 files changed, 268 insertions(+), 8 deletions(-) diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index ebe2c3a596a..04829b6ee45 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -2177,6 +2177,42 @@ deparseInsertSql(StringInfo buf, RangeTblEntry *rte, withCheckOptionList, returningList, retrieved_attrs); } +/* + * Build a COPY FROM STDIN statement using the TEXT format + */ +void +deparseCopySql(StringInfo buf, Relation rel, List *target_attrs) +{ + TupleDesc tupdesc = RelationGetDescr(rel); + bool first = true; + int nattrs = list_length(target_attrs); + + appendStringInfo(buf, "COPY "); + deparseRelation(buf, rel); + if (nattrs > 0) + appendStringInfo(buf, "("); + + foreach_int(attnum, target_attrs) + { + Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1); + + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoString(buf, ", "); + + first = false; + + appendStringInfoString(buf, quote_identifier(NameStr(attr->attname))); + } + if (nattrs > 0) + appendStringInfoString(buf, ") FROM STDIN"); + else + appendStringInfoString(buf, " FROM STDIN"); +} + + /* * rebuild remote INSERT statement * diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index 2ccb72c539a..22c2dcdb6b1 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -7603,6 +7603,27 @@ select * from grem1; (2 rows) delete from grem1; +-- test that fdw also use COPY FROM as a remote sql +set client_min_messages to 'log'; +create function insert_or_copy() returns trigger as $$ +declare query text; +begin + query := current_query(); + if query ~* '^COPY' then + raise notice 'COPY command'; + elsif query ~* '^INSERT' then + raise notice 'INSERT command'; + end if; +return new; +end; +$$ language plpgsql; +CREATE TRIGGER trig_row_before +BEFORE INSERT OR UPDATE OR DELETE ON gloc1 +FOR EACH ROW EXECUTE PROCEDURE insert_or_copy(); +copy grem1 from stdin; +LOG: received message via remote connection: NOTICE: COPY command +drop trigger trig_row_before on gloc1; +reset client_min_messages; -- test batch insert alter server loopback options (add batch_size '10'); explain (verbose, costs off) @@ -7620,16 +7641,18 @@ insert into grem1 (a) values (1), (2); select * from gloc1; a | b | c ---+---+--- + 3 | 6 | 1 | 2 | 2 | 4 | -(2 rows) +(3 rows) select * from grem1; a | b | c ---+---+--- + 3 | 6 | 9 1 | 2 | 3 2 | 4 | 6 -(2 rows) +(3 rows) delete from grem1; -- batch insert with foreign partitions. @@ -9544,7 +9567,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN COPY rem2, line 1: "-1 xyzzy" select * from rem2; f1 | f2 @@ -9701,7 +9725,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN COPY rem2 select * from rem2; f1 | f2 diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index 60d90329a65..6dd22e4043d 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -63,6 +63,9 @@ PG_MODULE_MAGIC_EXT( /* If no remote estimates, assume a sort costs 20% extra */ #define DEFAULT_FDW_SORT_MULTIPLIER 1.2 +/* Buffer size to send COPY IN data*/ +#define COPYBUFSIZ 8192 + /* * Indexes of FDW-private information stored in fdw_private lists. * @@ -198,6 +201,8 @@ typedef struct PgFdwModifyState bool has_returning; /* is there a RETURNING clause? */ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */ + bool use_copy; + /* info about parameters for prepared statement */ AttrNumber ctidAttno; /* attnum of input resjunk ctid column */ int p_nums; /* number of parameters to transmit */ @@ -545,6 +550,12 @@ static void merge_fdw_options(PgFdwRelationInfo *fpinfo, const PgFdwRelationInfo *fpinfo_o, const PgFdwRelationInfo *fpinfo_i); static int get_batch_size_option(Relation rel); +static TupleTableSlot **execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots); +static void convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot); /* @@ -2170,6 +2181,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, List *targetAttrs = NIL; List *retrieved_attrs = NIL; bool doNothing = false; + bool useCopy = false; /* * If the foreign table we are about to insert routed rows into is also an @@ -2247,11 +2259,41 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, rte = exec_rt_fetch(resultRelation, estate); } + /* + * We can use COPY for remote inserts only if all the following conditions + * are met: + * + * Direct Execution: The command is a COPY FROM on the foreign table + * itself, not part of a partitioned table's tuple routing. ( + * resultRelInfo->ri_RootResultRelInfo == NULL) + * + * No Local AFTER Triggers: There are no AFTER ROW triggers defined locally + * on the foreign table. + * + * Remote triggers might modify the inserted row. Because the COPY protocol + * does not support a RETURNING clause, we cannot retrieve those changes to + * synchronize the local TupleTableSlot required by local AFTER triggers. + */ + if (resultRelInfo->ri_RootResultRelInfo == NULL) + { + /* There is no RETURNING clause on COPY */ + Assert(resultRelInfo->ri_returningList == NIL); + + useCopy = (resultRelInfo->ri_TrigDesc == NULL || + !resultRelInfo->ri_TrigDesc->trig_insert_after_row); + } + /* Construct the SQL command string. */ - deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, - resultRelInfo->ri_WithCheckOptions, - resultRelInfo->ri_returningList, - &retrieved_attrs, &values_end_len); + if (useCopy) + { + deparseCopySql(&sql, rel, targetAttrs); + values_end_len = 0; /* Keep compiler quiet */ + } + else + deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, + resultRelInfo->ri_WithCheckOptions, + resultRelInfo->ri_returningList, + &retrieved_attrs, &values_end_len); /* Construct an execution state. */ fmstate = create_foreign_modify(mtstate->ps.state, @@ -2264,6 +2306,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, values_end_len, retrieved_attrs != NIL, retrieved_attrs); + fmstate->use_copy = useCopy; /* * If the given resultRelInfo already has PgFdwModifyState set, it means @@ -4093,6 +4136,9 @@ execute_foreign_modify(EState *estate, operation == CMD_UPDATE || operation == CMD_DELETE); + if (fmstate->use_copy) + return execute_foreign_modify_using_copy(fmstate, slots, numSlots); + /* First, process a pending asynchronous request, if any. */ if (fmstate->conn_state->pendingAreq) process_pending_request(fmstate->conn_state->pendingAreq); @@ -7886,3 +7932,128 @@ get_batch_size_option(Relation rel) return batch_size; } + +/* + * execute_foreign_modify_using_copy + * Perform foreign-table modification using the COPY command. + */ +static TupleTableSlot ** +execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots) +{ + PGresult *res; + StringInfoData copy_data; + int n_rows; + int i; + + Assert(fmstate->use_copy == true); + + /* Send COPY command */ + if (!PQsendQuery(fmstate->conn, fmstate->query)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* get the COPY result */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COPY_IN) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + /* Clean up the COPY command result */ + PQclear(res); + + /* Convert the TupleTableSlot data into a TEXT-formatted line */ + initStringInfo(©_data); + for (i = 0; i < *numSlots; i++) + { + convert_slot_to_copy_text(©_data, fmstate, slots[i]); + + /* + * Send initial COPY data if the buffer reaches the limit to avoid + * large memory usage. + */ + if (copy_data.len >= COPYBUFSIZ) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + resetStringInfo(©_data); + } + } + + /* Send the remaining COPY data */ + if (copy_data.len > 0) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + } + + pfree(copy_data.data); + + /* End the COPY operation */ + if (PQputCopyEnd(fmstate->conn, NULL) < 0 || PQflush(fmstate->conn)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* + * Get the result, and check for success. + */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + n_rows = atoi(PQcmdTuples(res)); + + /* And clean up */ + PQclear(res); + + MemoryContextReset(fmstate->temp_cxt); + + *numSlots = n_rows; + + /* + * Return NULL if nothing was inserted on the remote end + */ + return (n_rows > 0) ? slots : NULL; +} + +/* + * Write target attribute values from fmstate into buf buffer to be sent as + * COPY FROM STDIN data + */ +static void +convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot) +{ + TupleDesc tupdesc = RelationGetDescr(fmstate->rel); + bool first = true; + int i = 0; + + foreach_int(attnum, fmstate->target_attrs) + { + CompactAttribute *attr = TupleDescCompactAttr(tupdesc, attnum - 1); + Datum datum; + bool isnull; + + /* Ignore generated columns; they are set to DEFAULT */ + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoCharMacro(buf, '\t'); + first = false; + + datum = slot_getattr(slot, attnum, &isnull); + + if (isnull) + appendStringInfoString(buf, "\\N"); + else + { + const char *value = OutputFunctionCall(&fmstate->p_flinfo[i], + datum); + + appendStringInfoString(buf, value); + } + i++; + } + + appendStringInfoCharMacro(buf, '\n'); +} diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h index a2bb1ff352c..fc6922ddd4f 100644 --- a/contrib/postgres_fdw/postgres_fdw.h +++ b/contrib/postgres_fdw/postgres_fdw.h @@ -204,6 +204,7 @@ extern void rebuildInsertSql(StringInfo buf, Relation rel, char *orig_query, List *target_attrs, int values_end_len, int num_params, int num_rows); +extern void deparseCopySql(StringInfo buf, Relation rel, List *target_attrs); extern void deparseUpdateSql(StringInfo buf, RangeTblEntry *rte, Index rtindex, Relation rel, List *targetAttrs, diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql index 72d2d9c311b..90246ddbd02 100644 --- a/contrib/postgres_fdw/sql/postgres_fdw.sql +++ b/contrib/postgres_fdw/sql/postgres_fdw.sql @@ -1929,6 +1929,33 @@ select * from gloc1; select * from grem1; delete from grem1; +-- test that fdw also use COPY FROM as a remote sql +set client_min_messages to 'log'; + +create function insert_or_copy() returns trigger as $$ +declare query text; +begin + query := current_query(); + if query ~* '^COPY' then + raise notice 'COPY command'; + elsif query ~* '^INSERT' then + raise notice 'INSERT command'; + end if; +return new; +end; +$$ language plpgsql; + +CREATE TRIGGER trig_row_before +BEFORE INSERT OR UPDATE OR DELETE ON gloc1 +FOR EACH ROW EXECUTE PROCEDURE insert_or_copy(); + +copy grem1 from stdin; +3 +\. + +drop trigger trig_row_before on gloc1; +reset client_min_messages; + -- test batch insert alter server loopback options (add batch_size '10'); explain (verbose, costs off) -- 2.52.0 ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-05-27 23:22 Matheus Alcantara <[email protected]> parent: Matheus Alcantara <[email protected]> 1 sibling, 0 replies; 35+ messages in thread From: Matheus Alcantara @ 2026-05-27 23:22 UTC (permalink / raw) To: Masahiko Sawada <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers On Wed Apr 1, 2026 at 12:50 PM -03, Matheus Alcantara wrote: >> -- on local server >> copy t(a, d) from stdin; >> Enter data to be copied followed by a newline. >> End with a backslash and a period on a line by itself, or an EOF signal. >>>> 1 hello\nworld >>>> \. >> ERROR: invalid input syntax for type integer: "world" >> CONTEXT: COPY t, line 2, column a: "world" >> remote SQL command: COPY public.t(a, d) FROM STDIN (FORMAT TEXT) >> > > I think that we need something like CopyAttributeOutText() here. > > To fix this I've added appendStringInfoText() which is a similar version > of CopyAttributeOutText() that works with a StringInfo. I did not find > any function that I could reuse here, if such function exists please let > me know. > > I'm wondering if we should have this similar function or try to combine > both to avoid duplicated logic, although it looks complicated to me at > first look to combine these both usages. > I've spent some time hacking into this and actually I don't think that it's too complicated. Please see the attached patchset. 0001: Extract CopyAttributeOutText escape logic into a reusable function that can be used by postgres_fdw and by COPY TO routine. 0002: Is the main feature implementation using the TEXT format reusing the refactor from 0001 avoiding all the duplicated code introduced on this previous patch version. I've also removed the resultRelInfo->ri_WithCheckOptions == NIL check that I've introduced again by mistake since it was already commented at [1] that it should be removed. [1] https://www.postgresql.org/message-id/72ec1708-0c02-4ae9-b4f5-ee2bac5fd2f3%40gmail.com -- Matheus Alcantara EDB: https://www.enterprisedb.com From e02ecd11c93306a0160b7ffb87a00e1422b2bd8a Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Mon, 18 May 2026 19:23:43 -0300 Subject: [PATCH v15 1/2] Extract CopyEscapeText() for reuse outside COPY TO Refactor CopyAttributeOutText() to extract its core text escaping logic into a new public function CopyEscapeText() that operates on a StringInfo buffer with explicit parameters, removing the dependency on CopyToState. This enables other code paths, such as postgres_fdw, to reuse the COPY text format escaping logic without needing to construct a full CopyToState. CopyAttributeOutText() now becomes a thin wrapper that calls CopyEscapeText() with the appropriate values from CopyToState. Also introduce CopySendDataBuf() and CopySendCharBuf() macros as low-level helpers that operate directly on StringInfo buffers. Author: Matheus Alcantara <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/DDIZJ217OUDK.2R5WE4OGL5PTY%40gmail.com --- src/backend/commands/copyto.c | 85 ++++++++++++++++++++++++++--------- src/include/commands/copy.h | 6 +++ 2 files changed, 71 insertions(+), 20 deletions(-) diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index ffed63a2986..ceee0014cfc 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -553,6 +553,12 @@ SendCopyEnd(CopyToState cstate) pq_putemptymessage(PqMsg_CopyDone); } +#define CopySendCharBuf(buf, c) \ + appendStringInfoCharMacro(buf, c) + +#define CopySendDataBuf(buf, databuf, datasize) \ + appendBinaryStringInfo(buf, databuf, datasize) + /*---------- * CopySendData sends output data to the destination (file or frontend) * CopySendString does the same for null-terminated strings @@ -566,7 +572,7 @@ SendCopyEnd(CopyToState cstate) static void CopySendData(CopyToState cstate, const void *databuf, int datasize) { - appendBinaryStringInfo(cstate->fe_msgbuf, databuf, datasize); + CopySendDataBuf(cstate->fe_msgbuf, databuf, datasize); } static void @@ -578,7 +584,7 @@ CopySendString(CopyToState cstate, const char *str) static void CopySendChar(CopyToState cstate, char c) { - appendStringInfoCharMacro(cstate->fe_msgbuf, c); + CopySendCharBuf(cstate->fe_msgbuf, c); } static void @@ -1417,16 +1423,44 @@ CopyOneRowTo(CopyToState cstate, TupleTableSlot *slot) CopySendData(cstate, start, ptr - start); \ } while (0) -static void -CopyAttributeOutText(CopyToState cstate, const char *string) +/* Like above, but it works with a string buffer */ +#define DUMPSOFAR_TO_BUF() \ + do { \ + if (ptr > start) \ + CopySendDataBuf(buf, start, ptr - start); \ + } while (0) + + +/* + * Escape a string for COPY TEXT format output + * + * Escapes control characters, backslashes, and the delimiter character + * according to COPY TEXT format rules. The escaped string is appended + * to 'buf'. + * + * Parameters: + * buf - StringInfo buffer to append the escaped text to + * string - the input string to escape + * delimc - the delimiter character that must be escaped + * file_encoding - target encoding for the output + * need_transcoding - if true, convert from server encoding to file_encoding + * encoding_embeds_ascii - if true, the encoding may have ASCII bytes as + * non-first bytes of multi-byte characters + */ +void +CopyEscapeText(StringInfo buf, + const char *string, + char delimc, + int file_encoding, + bool need_transcoding, + bool encoding_embeds_ascii) { const char *ptr; const char *start; char c; - char delimc = cstate->opts.delim[0]; - if (cstate->need_transcoding) - ptr = pg_server_to_any(string, strlen(string), cstate->file_encoding); + if (need_transcoding) + ptr = pg_server_to_any(string, strlen(string), file_encoding); else ptr = string; @@ -1444,7 +1478,7 @@ CopyAttributeOutText(CopyToState cstate, const char *string) * it's worth making two copies of it to get the IS_HIGHBIT_SET() test out * of the normal safe-encoding path. */ - if (cstate->encoding_embeds_ascii) + if (encoding_embeds_ascii) { start = ptr; while ((c = *ptr) != '\0') @@ -1487,19 +1521,19 @@ CopyAttributeOutText(CopyToState cstate, const char *string) continue; /* fall to end of loop */ } /* if we get here, we need to convert the control char */ - DUMPSOFAR(); - CopySendChar(cstate, '\\'); - CopySendChar(cstate, c); + DUMPSOFAR_TO_BUF(); + CopySendCharBuf(buf, '\\'); + CopySendCharBuf(buf, c); start = ++ptr; /* do not include char in next run */ } else if (c == '\\' || c == delimc) { - DUMPSOFAR(); - CopySendChar(cstate, '\\'); + DUMPSOFAR_TO_BUF(); + CopySendCharBuf(buf, '\\'); start = ptr++; /* we include char in next run */ } else if (IS_HIGHBIT_SET(c)) - ptr += pg_encoding_mblen(cstate->file_encoding, ptr); + ptr += pg_encoding_mblen(file_encoding, ptr); else ptr++; } @@ -1547,15 +1581,15 @@ CopyAttributeOutText(CopyToState cstate, const char *string) continue; /* fall to end of loop */ } /* if we get here, we need to convert the control char */ - DUMPSOFAR(); - CopySendChar(cstate, '\\'); - CopySendChar(cstate, c); + DUMPSOFAR_TO_BUF(); + CopySendCharBuf(buf, '\\'); + CopySendCharBuf(buf, c); start = ++ptr; /* do not include char in next run */ } else if (c == '\\' || c == delimc) { - DUMPSOFAR(); - CopySendChar(cstate, '\\'); + DUMPSOFAR_TO_BUF(); + CopySendCharBuf(buf, '\\'); start = ptr++; /* we include char in next run */ } else @@ -1563,7 +1597,18 @@ CopyAttributeOutText(CopyToState cstate, const char *string) } } - DUMPSOFAR(); + DUMPSOFAR_TO_BUF(); +} + +static void +CopyAttributeOutText(CopyToState cstate, const char *string) +{ + CopyEscapeText(cstate->fe_msgbuf, + string, + cstate->opts.delim[0], + cstate->file_encoding, + cstate->need_transcoding, + cstate->encoding_embeds_ascii); } /* diff --git a/src/include/commands/copy.h b/src/include/commands/copy.h index abecfe51098..27ee58c7fdb 100644 --- a/src/include/commands/copy.h +++ b/src/include/commands/copy.h @@ -136,5 +136,11 @@ extern void EndCopyTo(CopyToState cstate); extern uint64 DoCopyTo(CopyToState cstate); extern List *CopyGetAttnums(TupleDesc tupDesc, Relation rel, List *attnamelist); +extern void CopyEscapeText(StringInfo buf, + const char *string, + char delimc, + int file_encoding, + bool need_transcoding, + bool encoding_embeds_ascii); #endif /* COPY_H */ -- 2.50.1 (Apple Git-155) From a4ce235442d38ebf751c4711bdd4b685db0c35ef Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Wed, 28 Jan 2026 19:55:48 -0300 Subject: [PATCH v15 2/2] postgres_fdw: Use COPY as remote SQL when possible When a user executes COPY on a foreign table, postgres_fdw previously sent INSERT statements to the foreign server. This commit enables using the COPY protocol instead, which is significantly faster for bulk inserts. The COPY protocol is used when copying data to a foreign table that has no triggers. Triggers are excluded because they might modify the inserted row, and since COPY does not support a RETURNING clause, we cannot synchronize the local TupleTableSlot with those changes for use in local AFTER triggers. This uses the CopyEscapeText() function introduced on 051b56b4f95 to properly escape text values for the COPY protocol. Author: Matheus Alcantara <[email protected]> Reviewed-By: Tomas Vondra <[email protected]> Reviewed-By: Jakub Wartak <[email protected]> Reviewed-By: jian he <[email protected]> Reviewed-By: Dewei Dai <[email protected]> Reviewed-By: Masahiko Sawada <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/DDIZJ217OUDK.2R5WE4OGL5PTY%40gmail.com --- contrib/postgres_fdw/deparse.c | 56 +++++ .../postgres_fdw/expected/postgres_fdw.out | 75 ++++++- contrib/postgres_fdw/postgres_fdw.c | 202 +++++++++++++++++- contrib/postgres_fdw/postgres_fdw.h | 1 + contrib/postgres_fdw/sql/postgres_fdw.sql | 82 +++++++ 5 files changed, 407 insertions(+), 9 deletions(-) diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index 2dcc6c8af1b..b34792e7dc1 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -2177,6 +2177,62 @@ deparseInsertSql(StringInfo buf, RangeTblEntry *rte, withCheckOptionList, returningList, retrieved_attrs); } +/* + * Build a COPY FROM STDIN statement using the TEXT format + */ +void +deparseCopySql(StringInfo buf, Relation rel, List *target_attrs) +{ + Oid relid = RelationGetRelid(rel); + TupleDesc tupdesc = RelationGetDescr(rel); + bool first = true; + int nattrs = list_length(target_attrs); + + appendStringInfo(buf, "COPY "); + deparseRelation(buf, rel); + if (nattrs > 0) + appendStringInfoChar(buf, '('); + + foreach_int(attnum, target_attrs) + { + Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1); + char *colname; + List *options; + ListCell *lc; + + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoString(buf, ", "); + + first = false; + + /* Use attribute name or column_name option. */ + colname = NameStr(attr->attname); + options = GetForeignColumnOptions(relid, attnum); + foreach(lc, options) + { + DefElem *def = (DefElem *) lfirst(lc); + + if (strcmp(def->defname, "column_name") == 0) + { + colname = defGetString(def); + break; + } + } + + appendStringInfoString(buf, quote_identifier(colname)); + } + if (nattrs > 0) + appendStringInfoString(buf, ") FROM STDIN"); + else + appendStringInfoString(buf, " FROM STDIN"); + + appendStringInfoString(buf, " (FORMAT TEXT)"); +} + + /* * rebuild remote INSERT statement * diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index e90289e4ab1..ae655545074 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -7654,6 +7654,28 @@ select * from grem1; (2 rows) delete from grem1; +-- test that fdw also use COPY FROM as a remote sql +set client_min_messages to 'log'; +create function insert_or_copy() returns trigger as $$ +declare query text; +begin + query := current_query(); + raise notice '%', query; +return new; +end; +$$ language plpgsql; +CREATE TRIGGER trig_row_before +BEFORE INSERT OR UPDATE OR DELETE ON gloc1 +FOR EACH ROW EXECUTE PROCEDURE insert_or_copy(); +copy grem1 from stdin; +LOG: received message via remote connection: NOTICE: COPY public.gloc1(a) FROM STDIN (FORMAT TEXT) +drop trigger trig_row_before on gloc1; +reset client_min_messages; +-- test that copy does not fail with column_name alias +create table gloc2(xxx int); +create foreign table grem2(a int) server loopback options(table_name 'gloc2'); +alter foreign table grem2 alter column a options (column_name 'xxx'); +copy grem2 from stdin; -- test batch insert alter server loopback options (add batch_size '10'); explain (verbose, costs off) @@ -7671,16 +7693,18 @@ insert into grem1 (a) values (1), (2); select * from gloc1; a | b | c ---+---+--- + 3 | 6 | 1 | 2 | 2 | 4 | -(2 rows) +(3 rows) select * from grem1; a | b | c ---+---+--- + 3 | 6 | 9 1 | 2 | 3 2 | 4 | 6 -(2 rows) +(3 rows) delete from grem1; -- batch insert with foreign partitions. @@ -7705,6 +7729,12 @@ select count(*) from tab_batch_sharded; drop table tab_batch_local; drop table tab_batch_sharded; drop table tab_batch_sharded_p1_remote; +-- test batch insert using copy +set client_min_messages to 'debug1'; +copy grem1 from stdin; +DEBUG: foreign modify with COPY batch_size: 10 +DEBUG: foreign modify with COPY batch_size: 10 +reset client_min_messages; alter server loopback options (drop batch_size); -- =================================================================== -- test local triggers @@ -9595,7 +9625,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN (FORMAT TEXT) COPY rem2, line 1: "-1 xyzzy" select * from rem2; f1 | f2 @@ -9752,7 +9783,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN (FORMAT TEXT) COPY rem2 select * from rem2; f1 | f2 @@ -9791,6 +9823,41 @@ select * from rem2; drop trigger trig_null on loc2; delete from rem2; +-- Test COPY FROM with column list and special characters +copy rem2 (f1, f2) from stdin; +select * from rem2; + f1 | f2 +----+------- + 1 | hello+ + | world +(1 row) + +delete from rem2; +-- Test COPY with NULL and special characters +copy rem2 from stdin; +select * from rem2; + f1 | f2 +----+----- + 1 | + | bar + 3 | a"b +(3 rows) + +delete from rem2; +-- Test that float numbers do not loose precision when sending to the foreign +-- server +create table f(a float); +create foreign table f_fdw(a float) server loopback options(table_name 'f'); +set extra_float_digits = 0; +copy f_fdw from stdin; +reset extra_float_digits; +select * from f; + a +-------------------- + 1.0000000000000002 +(1 row) + +drop table f; -- Check with zero-column foreign table; batch insert will be disabled alter table loc2 drop column f1; alter table loc2 drop column f2; diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index 0a589f8db74..7bde40ffe92 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -18,6 +18,7 @@ #include "access/sysattr.h" #include "access/table.h" #include "catalog/pg_opfamily.h" +#include "commands/copy.h" #include "commands/defrem.h" #include "commands/explain_format.h" #include "commands/explain_state.h" @@ -27,6 +28,7 @@ #include "executor/spi.h" #include "foreign/fdwapi.h" #include "funcapi.h" +#include "mb/pg_wchar.h" #include "miscadmin.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -67,6 +69,9 @@ PG_MODULE_MAGIC_EXT( /* If no remote estimates, assume a sort costs 20% extra */ #define DEFAULT_FDW_SORT_MULTIPLIER 1.2 +/* Buffer size to send COPY IN data*/ +#define COPYBUFSIZ 8192 + /* * Indexes of FDW-private information stored in fdw_private lists. * @@ -202,6 +207,8 @@ typedef struct PgFdwModifyState bool has_returning; /* is there a RETURNING clause? */ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */ + bool use_copy; /* use COPY protocol for ExecForeignInsert? */ + /* info about parameters for prepared statement */ AttrNumber ctidAttno; /* attnum of input resjunk ctid column */ int p_nums; /* number of parameters to transmit */ @@ -760,6 +767,13 @@ static void merge_fdw_options(PgFdwRelationInfo *fpinfo, const PgFdwRelationInfo *fpinfo_o, const PgFdwRelationInfo *fpinfo_i); static int get_batch_size_option(Relation rel); +static TupleTableSlot **execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots); +static void convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot); +static void appendStringInfoText(StringInfo buf, const char *string); /* @@ -2381,11 +2395,12 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, RangeTblEntry *rte; TupleDesc tupdesc = RelationGetDescr(rel); int attnum; - int values_end_len; + int values_end_len = 0; StringInfoData sql; List *targetAttrs = NIL; List *retrieved_attrs = NIL; bool doNothing = false; + bool useCopy = false; /* * If the foreign table we are about to insert routed rows into is also an @@ -2463,11 +2478,38 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, rte = exec_rt_fetch(resultRelation, estate); } + /* + * We can use COPY for remote inserts only if all the following conditions + * are met: + * + * Direct Execution: The command is a COPY FROM on the foreign table + * itself, not part of a partitioned table's tuple routing. + * + * No Local AFTER Triggers: There are no AFTER ROW triggers defined + * locally on the foreign table. + * + * Remote triggers might modify the inserted row. Because the COPY + * protocol does not support a RETURNING clause, we cannot retrieve those + * changes to synchronize the local TupleTableSlot required by local AFTER + * triggers. + */ + if (resultRelInfo->ri_RootResultRelInfo == NULL) + { + /* There is no RETURNING clause on COPY */ + Assert(resultRelInfo->ri_returningList == NIL); + + useCopy = (resultRelInfo->ri_TrigDesc == NULL || + !resultRelInfo->ri_TrigDesc->trig_insert_after_row); + } + /* Construct the SQL command string. */ - deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, - resultRelInfo->ri_WithCheckOptions, - resultRelInfo->ri_returningList, - &retrieved_attrs, &values_end_len); + if (useCopy) + deparseCopySql(&sql, rel, targetAttrs); + else + deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, + resultRelInfo->ri_WithCheckOptions, + resultRelInfo->ri_returningList, + &retrieved_attrs, &values_end_len); /* Construct an execution state. */ fmstate = create_foreign_modify(mtstate->ps.state, @@ -2480,6 +2522,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, values_end_len, retrieved_attrs != NIL, retrieved_attrs); + fmstate->use_copy = useCopy; /* * If the given resultRelInfo already has PgFdwModifyState set, it means @@ -4309,6 +4352,9 @@ execute_foreign_modify(EState *estate, operation == CMD_UPDATE || operation == CMD_DELETE); + if (fmstate->use_copy) + return execute_foreign_modify_using_copy(fmstate, slots, numSlots); + /* First, process a pending asynchronous request, if any. */ if (fmstate->conn_state->pendingAreq) process_pending_request(fmstate->conn_state->pendingAreq); @@ -8835,3 +8881,149 @@ get_batch_size_option(Relation rel) return batch_size; } + +/* + * execute_foreign_modify_using_copy + * Perform foreign-table modification using the COPY command. + */ +static TupleTableSlot ** +execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots) +{ + PGresult *res; + StringInfoData copy_data; + int n_rows; + int i; + int nestlevel; + + Assert(fmstate->use_copy == true); + + elog(DEBUG1, "foreign modify with COPY batch_size: %d", fmstate->batch_size); + + /* Make sure any constants in the slots are printed portably */ + nestlevel = set_transmission_modes(); + + /* Send COPY command */ + if (!PQsendQuery(fmstate->conn, fmstate->query)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* get the COPY result */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COPY_IN) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + /* Clean up the COPY command result */ + PQclear(res); + + /* Convert the TupleTableSlot data into a TEXT-formatted line */ + initStringInfo(©_data); + for (i = 0; i < *numSlots; i++) + { + convert_slot_to_copy_text(©_data, fmstate, slots[i]); + + /* + * Send initial COPY data if the buffer reaches the limit to avoid + * large memory usage. + */ + if (copy_data.len >= COPYBUFSIZ) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + resetStringInfo(©_data); + } + } + + /* Send the remaining COPY data */ + if (copy_data.len > 0) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + } + + pfree(copy_data.data); + + /* End the COPY operation */ + if (PQputCopyEnd(fmstate->conn, NULL) < 0 || PQflush(fmstate->conn)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* + * Get the result, and check for success. + */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + n_rows = atoi(PQcmdTuples(res)); + + /* And clean up */ + PQclear(res); + + reset_transmission_modes(nestlevel); + + MemoryContextReset(fmstate->temp_cxt); + + *numSlots = n_rows; + + /* + * Return NULL if nothing was inserted on the remote end + */ + return (n_rows > 0) ? slots : NULL; +} + +/* + * Write target attribute values from fmstate into buf buffer to be sent as + * COPY FROM STDIN data + */ +static void +convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot) +{ + TupleDesc tupdesc = RelationGetDescr(fmstate->rel); + bool first = true; + int i = 0; + + foreach_int(attnum, fmstate->target_attrs) + { + CompactAttribute *attr = TupleDescCompactAttr(tupdesc, attnum - 1); + Datum datum; + bool isnull; + + /* Ignore generated columns; they are set to DEFAULT */ + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoCharMacro(buf, '\t'); + first = false; + + datum = slot_getattr(slot, attnum, &isnull); + + if (isnull) + appendStringInfoString(buf, "\\N"); + else + { + const char *value = OutputFunctionCall(&fmstate->p_flinfo[i], + datum); + + /* Escape the value if needed */ + appendStringInfoText(buf, value); + } + i++; + } + + appendStringInfoCharMacro(buf, '\n'); +} + +/* Append a string to buf, escaping special characters for COPY TEXT format. */ +static void +appendStringInfoText(StringInfo buf, const char *string) +{ + CopyEscapeText(buf, + string, + '\t', + PG_UTF8, + false, + false); +} diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h index a2bb1ff352c..fc6922ddd4f 100644 --- a/contrib/postgres_fdw/postgres_fdw.h +++ b/contrib/postgres_fdw/postgres_fdw.h @@ -204,6 +204,7 @@ extern void rebuildInsertSql(StringInfo buf, Relation rel, char *orig_query, List *target_attrs, int values_end_len, int num_params, int num_rows); +extern void deparseCopySql(StringInfo buf, Relation rel, List *target_attrs); extern void deparseUpdateSql(StringInfo buf, RangeTblEntry *rte, Index rtindex, Relation rel, List *targetAttrs, diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql index dfc58beb0d2..db95b148ca8 100644 --- a/contrib/postgres_fdw/sql/postgres_fdw.sql +++ b/contrib/postgres_fdw/sql/postgres_fdw.sql @@ -1970,6 +1970,37 @@ select * from gloc1; select * from grem1; delete from grem1; +-- test that fdw also use COPY FROM as a remote sql +set client_min_messages to 'log'; + +create function insert_or_copy() returns trigger as $$ +declare query text; +begin + query := current_query(); + raise notice '%', query; +return new; +end; +$$ language plpgsql; + +CREATE TRIGGER trig_row_before +BEFORE INSERT OR UPDATE OR DELETE ON gloc1 +FOR EACH ROW EXECUTE PROCEDURE insert_or_copy(); + +copy grem1 from stdin; +3 +\. + +drop trigger trig_row_before on gloc1; +reset client_min_messages; + +-- test that copy does not fail with column_name alias +create table gloc2(xxx int); +create foreign table grem2(a int) server loopback options(table_name 'gloc2'); +alter foreign table grem2 alter column a options (column_name 'xxx'); +copy grem2 from stdin; +1 +\. + -- test batch insert alter server loopback options (add batch_size '10'); explain (verbose, costs off) @@ -1996,6 +2027,24 @@ drop table tab_batch_local; drop table tab_batch_sharded; drop table tab_batch_sharded_p1_remote; +-- test batch insert using copy +set client_min_messages to 'debug1'; +copy grem1 from stdin; +1 +2 +3 +4 +5 +6 +7 +8 +9 +10 +11 +12 +\. +reset client_min_messages; + alter server loopback options (drop batch_size); -- =================================================================== @@ -3060,6 +3109,39 @@ drop trigger trig_null on loc2; delete from rem2; +-- Test COPY FROM with column list and special characters +copy rem2 (f1, f2) from stdin; +1 hello\nworld +\. +select * from rem2; + +delete from rem2; + +-- Test COPY with NULL and special characters +copy rem2 from stdin; +1 \N +\N bar +3 a"b +\. +select * from rem2; + +delete from rem2; + +-- Test that float numbers do not loose precision when sending to the foreign +-- server +create table f(a float); +create foreign table f_fdw(a float) server loopback options(table_name 'f'); + +set extra_float_digits = 0; +copy f_fdw from stdin; +1.0000000000000002 +\. + +reset extra_float_digits; +select * from f; + +drop table f; + -- Check with zero-column foreign table; batch insert will be disabled alter table loc2 drop column f1; alter table loc2 drop column f2; -- 2.50.1 (Apple Git-155) Attachments: [text/plain] v15-0001-Extract-CopyEscapeText-for-reuse-outside-COPY-TO.patch (6.5K, ../../[email protected]/2-v15-0001-Extract-CopyEscapeText-for-reuse-outside-COPY-TO.patch) download | inline diff: From e02ecd11c93306a0160b7ffb87a00e1422b2bd8a Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Mon, 18 May 2026 19:23:43 -0300 Subject: [PATCH v15 1/2] Extract CopyEscapeText() for reuse outside COPY TO Refactor CopyAttributeOutText() to extract its core text escaping logic into a new public function CopyEscapeText() that operates on a StringInfo buffer with explicit parameters, removing the dependency on CopyToState. This enables other code paths, such as postgres_fdw, to reuse the COPY text format escaping logic without needing to construct a full CopyToState. CopyAttributeOutText() now becomes a thin wrapper that calls CopyEscapeText() with the appropriate values from CopyToState. Also introduce CopySendDataBuf() and CopySendCharBuf() macros as low-level helpers that operate directly on StringInfo buffers. Author: Matheus Alcantara <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/DDIZJ217OUDK.2R5WE4OGL5PTY%40gmail.com --- src/backend/commands/copyto.c | 85 ++++++++++++++++++++++++++--------- src/include/commands/copy.h | 6 +++ 2 files changed, 71 insertions(+), 20 deletions(-) diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index ffed63a2986..ceee0014cfc 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -553,6 +553,12 @@ SendCopyEnd(CopyToState cstate) pq_putemptymessage(PqMsg_CopyDone); } +#define CopySendCharBuf(buf, c) \ + appendStringInfoCharMacro(buf, c) + +#define CopySendDataBuf(buf, databuf, datasize) \ + appendBinaryStringInfo(buf, databuf, datasize) + /*---------- * CopySendData sends output data to the destination (file or frontend) * CopySendString does the same for null-terminated strings @@ -566,7 +572,7 @@ SendCopyEnd(CopyToState cstate) static void CopySendData(CopyToState cstate, const void *databuf, int datasize) { - appendBinaryStringInfo(cstate->fe_msgbuf, databuf, datasize); + CopySendDataBuf(cstate->fe_msgbuf, databuf, datasize); } static void @@ -578,7 +584,7 @@ CopySendString(CopyToState cstate, const char *str) static void CopySendChar(CopyToState cstate, char c) { - appendStringInfoCharMacro(cstate->fe_msgbuf, c); + CopySendCharBuf(cstate->fe_msgbuf, c); } static void @@ -1417,16 +1423,44 @@ CopyOneRowTo(CopyToState cstate, TupleTableSlot *slot) CopySendData(cstate, start, ptr - start); \ } while (0) -static void -CopyAttributeOutText(CopyToState cstate, const char *string) +/* Like above, but it works with a string buffer */ +#define DUMPSOFAR_TO_BUF() \ + do { \ + if (ptr > start) \ + CopySendDataBuf(buf, start, ptr - start); \ + } while (0) + + +/* + * Escape a string for COPY TEXT format output + * + * Escapes control characters, backslashes, and the delimiter character + * according to COPY TEXT format rules. The escaped string is appended + * to 'buf'. + * + * Parameters: + * buf - StringInfo buffer to append the escaped text to + * string - the input string to escape + * delimc - the delimiter character that must be escaped + * file_encoding - target encoding for the output + * need_transcoding - if true, convert from server encoding to file_encoding + * encoding_embeds_ascii - if true, the encoding may have ASCII bytes as + * non-first bytes of multi-byte characters + */ +void +CopyEscapeText(StringInfo buf, + const char *string, + char delimc, + int file_encoding, + bool need_transcoding, + bool encoding_embeds_ascii) { const char *ptr; const char *start; char c; - char delimc = cstate->opts.delim[0]; - if (cstate->need_transcoding) - ptr = pg_server_to_any(string, strlen(string), cstate->file_encoding); + if (need_transcoding) + ptr = pg_server_to_any(string, strlen(string), file_encoding); else ptr = string; @@ -1444,7 +1478,7 @@ CopyAttributeOutText(CopyToState cstate, const char *string) * it's worth making two copies of it to get the IS_HIGHBIT_SET() test out * of the normal safe-encoding path. */ - if (cstate->encoding_embeds_ascii) + if (encoding_embeds_ascii) { start = ptr; while ((c = *ptr) != '\0') @@ -1487,19 +1521,19 @@ CopyAttributeOutText(CopyToState cstate, const char *string) continue; /* fall to end of loop */ } /* if we get here, we need to convert the control char */ - DUMPSOFAR(); - CopySendChar(cstate, '\\'); - CopySendChar(cstate, c); + DUMPSOFAR_TO_BUF(); + CopySendCharBuf(buf, '\\'); + CopySendCharBuf(buf, c); start = ++ptr; /* do not include char in next run */ } else if (c == '\\' || c == delimc) { - DUMPSOFAR(); - CopySendChar(cstate, '\\'); + DUMPSOFAR_TO_BUF(); + CopySendCharBuf(buf, '\\'); start = ptr++; /* we include char in next run */ } else if (IS_HIGHBIT_SET(c)) - ptr += pg_encoding_mblen(cstate->file_encoding, ptr); + ptr += pg_encoding_mblen(file_encoding, ptr); else ptr++; } @@ -1547,15 +1581,15 @@ CopyAttributeOutText(CopyToState cstate, const char *string) continue; /* fall to end of loop */ } /* if we get here, we need to convert the control char */ - DUMPSOFAR(); - CopySendChar(cstate, '\\'); - CopySendChar(cstate, c); + DUMPSOFAR_TO_BUF(); + CopySendCharBuf(buf, '\\'); + CopySendCharBuf(buf, c); start = ++ptr; /* do not include char in next run */ } else if (c == '\\' || c == delimc) { - DUMPSOFAR(); - CopySendChar(cstate, '\\'); + DUMPSOFAR_TO_BUF(); + CopySendCharBuf(buf, '\\'); start = ptr++; /* we include char in next run */ } else @@ -1563,7 +1597,18 @@ CopyAttributeOutText(CopyToState cstate, const char *string) } } - DUMPSOFAR(); + DUMPSOFAR_TO_BUF(); +} + +static void +CopyAttributeOutText(CopyToState cstate, const char *string) +{ + CopyEscapeText(cstate->fe_msgbuf, + string, + cstate->opts.delim[0], + cstate->file_encoding, + cstate->need_transcoding, + cstate->encoding_embeds_ascii); } /* diff --git a/src/include/commands/copy.h b/src/include/commands/copy.h index abecfe51098..27ee58c7fdb 100644 --- a/src/include/commands/copy.h +++ b/src/include/commands/copy.h @@ -136,5 +136,11 @@ extern void EndCopyTo(CopyToState cstate); extern uint64 DoCopyTo(CopyToState cstate); extern List *CopyGetAttnums(TupleDesc tupDesc, Relation rel, List *attnamelist); +extern void CopyEscapeText(StringInfo buf, + const char *string, + char delimc, + int file_encoding, + bool need_transcoding, + bool encoding_embeds_ascii); #endif /* COPY_H */ -- 2.50.1 (Apple Git-155) [text/plain] v15-0002-postgres_fdw-Use-COPY-as-remote-SQL-when-possibl.patch (18.4K, ../../[email protected]/3-v15-0002-postgres_fdw-Use-COPY-as-remote-SQL-when-possibl.patch) download | inline diff: From a4ce235442d38ebf751c4711bdd4b685db0c35ef Mon Sep 17 00:00:00 2001 From: Matheus Alcantara <[email protected]> Date: Wed, 28 Jan 2026 19:55:48 -0300 Subject: [PATCH v15 2/2] postgres_fdw: Use COPY as remote SQL when possible When a user executes COPY on a foreign table, postgres_fdw previously sent INSERT statements to the foreign server. This commit enables using the COPY protocol instead, which is significantly faster for bulk inserts. The COPY protocol is used when copying data to a foreign table that has no triggers. Triggers are excluded because they might modify the inserted row, and since COPY does not support a RETURNING clause, we cannot synchronize the local TupleTableSlot with those changes for use in local AFTER triggers. This uses the CopyEscapeText() function introduced on 051b56b4f95 to properly escape text values for the COPY protocol. Author: Matheus Alcantara <[email protected]> Reviewed-By: Tomas Vondra <[email protected]> Reviewed-By: Jakub Wartak <[email protected]> Reviewed-By: jian he <[email protected]> Reviewed-By: Dewei Dai <[email protected]> Reviewed-By: Masahiko Sawada <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/DDIZJ217OUDK.2R5WE4OGL5PTY%40gmail.com --- contrib/postgres_fdw/deparse.c | 56 +++++ .../postgres_fdw/expected/postgres_fdw.out | 75 ++++++- contrib/postgres_fdw/postgres_fdw.c | 202 +++++++++++++++++- contrib/postgres_fdw/postgres_fdw.h | 1 + contrib/postgres_fdw/sql/postgres_fdw.sql | 82 +++++++ 5 files changed, 407 insertions(+), 9 deletions(-) diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index 2dcc6c8af1b..b34792e7dc1 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -2177,6 +2177,62 @@ deparseInsertSql(StringInfo buf, RangeTblEntry *rte, withCheckOptionList, returningList, retrieved_attrs); } +/* + * Build a COPY FROM STDIN statement using the TEXT format + */ +void +deparseCopySql(StringInfo buf, Relation rel, List *target_attrs) +{ + Oid relid = RelationGetRelid(rel); + TupleDesc tupdesc = RelationGetDescr(rel); + bool first = true; + int nattrs = list_length(target_attrs); + + appendStringInfo(buf, "COPY "); + deparseRelation(buf, rel); + if (nattrs > 0) + appendStringInfoChar(buf, '('); + + foreach_int(attnum, target_attrs) + { + Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1); + char *colname; + List *options; + ListCell *lc; + + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoString(buf, ", "); + + first = false; + + /* Use attribute name or column_name option. */ + colname = NameStr(attr->attname); + options = GetForeignColumnOptions(relid, attnum); + foreach(lc, options) + { + DefElem *def = (DefElem *) lfirst(lc); + + if (strcmp(def->defname, "column_name") == 0) + { + colname = defGetString(def); + break; + } + } + + appendStringInfoString(buf, quote_identifier(colname)); + } + if (nattrs > 0) + appendStringInfoString(buf, ") FROM STDIN"); + else + appendStringInfoString(buf, " FROM STDIN"); + + appendStringInfoString(buf, " (FORMAT TEXT)"); +} + + /* * rebuild remote INSERT statement * diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index e90289e4ab1..ae655545074 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -7654,6 +7654,28 @@ select * from grem1; (2 rows) delete from grem1; +-- test that fdw also use COPY FROM as a remote sql +set client_min_messages to 'log'; +create function insert_or_copy() returns trigger as $$ +declare query text; +begin + query := current_query(); + raise notice '%', query; +return new; +end; +$$ language plpgsql; +CREATE TRIGGER trig_row_before +BEFORE INSERT OR UPDATE OR DELETE ON gloc1 +FOR EACH ROW EXECUTE PROCEDURE insert_or_copy(); +copy grem1 from stdin; +LOG: received message via remote connection: NOTICE: COPY public.gloc1(a) FROM STDIN (FORMAT TEXT) +drop trigger trig_row_before on gloc1; +reset client_min_messages; +-- test that copy does not fail with column_name alias +create table gloc2(xxx int); +create foreign table grem2(a int) server loopback options(table_name 'gloc2'); +alter foreign table grem2 alter column a options (column_name 'xxx'); +copy grem2 from stdin; -- test batch insert alter server loopback options (add batch_size '10'); explain (verbose, costs off) @@ -7671,16 +7693,18 @@ insert into grem1 (a) values (1), (2); select * from gloc1; a | b | c ---+---+--- + 3 | 6 | 1 | 2 | 2 | 4 | -(2 rows) +(3 rows) select * from grem1; a | b | c ---+---+--- + 3 | 6 | 9 1 | 2 | 3 2 | 4 | 6 -(2 rows) +(3 rows) delete from grem1; -- batch insert with foreign partitions. @@ -7705,6 +7729,12 @@ select count(*) from tab_batch_sharded; drop table tab_batch_local; drop table tab_batch_sharded; drop table tab_batch_sharded_p1_remote; +-- test batch insert using copy +set client_min_messages to 'debug1'; +copy grem1 from stdin; +DEBUG: foreign modify with COPY batch_size: 10 +DEBUG: foreign modify with COPY batch_size: 10 +reset client_min_messages; alter server loopback options (drop batch_size); -- =================================================================== -- test local triggers @@ -9595,7 +9625,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN (FORMAT TEXT) COPY rem2, line 1: "-1 xyzzy" select * from rem2; f1 | f2 @@ -9752,7 +9783,8 @@ copy rem2 from stdin; copy rem2 from stdin; -- ERROR ERROR: new row for relation "loc2" violates check constraint "loc2_f1positive" DETAIL: Failing row contains (-1, xyzzy). -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN (FORMAT TEXT) COPY rem2 select * from rem2; f1 | f2 @@ -9791,6 +9823,41 @@ select * from rem2; drop trigger trig_null on loc2; delete from rem2; +-- Test COPY FROM with column list and special characters +copy rem2 (f1, f2) from stdin; +select * from rem2; + f1 | f2 +----+------- + 1 | hello+ + | world +(1 row) + +delete from rem2; +-- Test COPY with NULL and special characters +copy rem2 from stdin; +select * from rem2; + f1 | f2 +----+----- + 1 | + | bar + 3 | a"b +(3 rows) + +delete from rem2; +-- Test that float numbers do not loose precision when sending to the foreign +-- server +create table f(a float); +create foreign table f_fdw(a float) server loopback options(table_name 'f'); +set extra_float_digits = 0; +copy f_fdw from stdin; +reset extra_float_digits; +select * from f; + a +-------------------- + 1.0000000000000002 +(1 row) + +drop table f; -- Check with zero-column foreign table; batch insert will be disabled alter table loc2 drop column f1; alter table loc2 drop column f2; diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index 0a589f8db74..7bde40ffe92 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -18,6 +18,7 @@ #include "access/sysattr.h" #include "access/table.h" #include "catalog/pg_opfamily.h" +#include "commands/copy.h" #include "commands/defrem.h" #include "commands/explain_format.h" #include "commands/explain_state.h" @@ -27,6 +28,7 @@ #include "executor/spi.h" #include "foreign/fdwapi.h" #include "funcapi.h" +#include "mb/pg_wchar.h" #include "miscadmin.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -67,6 +69,9 @@ PG_MODULE_MAGIC_EXT( /* If no remote estimates, assume a sort costs 20% extra */ #define DEFAULT_FDW_SORT_MULTIPLIER 1.2 +/* Buffer size to send COPY IN data*/ +#define COPYBUFSIZ 8192 + /* * Indexes of FDW-private information stored in fdw_private lists. * @@ -202,6 +207,8 @@ typedef struct PgFdwModifyState bool has_returning; /* is there a RETURNING clause? */ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */ + bool use_copy; /* use COPY protocol for ExecForeignInsert? */ + /* info about parameters for prepared statement */ AttrNumber ctidAttno; /* attnum of input resjunk ctid column */ int p_nums; /* number of parameters to transmit */ @@ -760,6 +767,13 @@ static void merge_fdw_options(PgFdwRelationInfo *fpinfo, const PgFdwRelationInfo *fpinfo_o, const PgFdwRelationInfo *fpinfo_i); static int get_batch_size_option(Relation rel); +static TupleTableSlot **execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots); +static void convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot); +static void appendStringInfoText(StringInfo buf, const char *string); /* @@ -2381,11 +2395,12 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, RangeTblEntry *rte; TupleDesc tupdesc = RelationGetDescr(rel); int attnum; - int values_end_len; + int values_end_len = 0; StringInfoData sql; List *targetAttrs = NIL; List *retrieved_attrs = NIL; bool doNothing = false; + bool useCopy = false; /* * If the foreign table we are about to insert routed rows into is also an @@ -2463,11 +2478,38 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, rte = exec_rt_fetch(resultRelation, estate); } + /* + * We can use COPY for remote inserts only if all the following conditions + * are met: + * + * Direct Execution: The command is a COPY FROM on the foreign table + * itself, not part of a partitioned table's tuple routing. + * + * No Local AFTER Triggers: There are no AFTER ROW triggers defined + * locally on the foreign table. + * + * Remote triggers might modify the inserted row. Because the COPY + * protocol does not support a RETURNING clause, we cannot retrieve those + * changes to synchronize the local TupleTableSlot required by local AFTER + * triggers. + */ + if (resultRelInfo->ri_RootResultRelInfo == NULL) + { + /* There is no RETURNING clause on COPY */ + Assert(resultRelInfo->ri_returningList == NIL); + + useCopy = (resultRelInfo->ri_TrigDesc == NULL || + !resultRelInfo->ri_TrigDesc->trig_insert_after_row); + } + /* Construct the SQL command string. */ - deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, - resultRelInfo->ri_WithCheckOptions, - resultRelInfo->ri_returningList, - &retrieved_attrs, &values_end_len); + if (useCopy) + deparseCopySql(&sql, rel, targetAttrs); + else + deparseInsertSql(&sql, rte, resultRelation, rel, targetAttrs, doNothing, + resultRelInfo->ri_WithCheckOptions, + resultRelInfo->ri_returningList, + &retrieved_attrs, &values_end_len); /* Construct an execution state. */ fmstate = create_foreign_modify(mtstate->ps.state, @@ -2480,6 +2522,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate, values_end_len, retrieved_attrs != NIL, retrieved_attrs); + fmstate->use_copy = useCopy; /* * If the given resultRelInfo already has PgFdwModifyState set, it means @@ -4309,6 +4352,9 @@ execute_foreign_modify(EState *estate, operation == CMD_UPDATE || operation == CMD_DELETE); + if (fmstate->use_copy) + return execute_foreign_modify_using_copy(fmstate, slots, numSlots); + /* First, process a pending asynchronous request, if any. */ if (fmstate->conn_state->pendingAreq) process_pending_request(fmstate->conn_state->pendingAreq); @@ -8835,3 +8881,149 @@ get_batch_size_option(Relation rel) return batch_size; } + +/* + * execute_foreign_modify_using_copy + * Perform foreign-table modification using the COPY command. + */ +static TupleTableSlot ** +execute_foreign_modify_using_copy(PgFdwModifyState *fmstate, + TupleTableSlot **slots, + int *numSlots) +{ + PGresult *res; + StringInfoData copy_data; + int n_rows; + int i; + int nestlevel; + + Assert(fmstate->use_copy == true); + + elog(DEBUG1, "foreign modify with COPY batch_size: %d", fmstate->batch_size); + + /* Make sure any constants in the slots are printed portably */ + nestlevel = set_transmission_modes(); + + /* Send COPY command */ + if (!PQsendQuery(fmstate->conn, fmstate->query)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* get the COPY result */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COPY_IN) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + /* Clean up the COPY command result */ + PQclear(res); + + /* Convert the TupleTableSlot data into a TEXT-formatted line */ + initStringInfo(©_data); + for (i = 0; i < *numSlots; i++) + { + convert_slot_to_copy_text(©_data, fmstate, slots[i]); + + /* + * Send initial COPY data if the buffer reaches the limit to avoid + * large memory usage. + */ + if (copy_data.len >= COPYBUFSIZ) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + resetStringInfo(©_data); + } + } + + /* Send the remaining COPY data */ + if (copy_data.len > 0) + { + if (PQputCopyData(fmstate->conn, copy_data.data, copy_data.len) <= 0) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + } + + pfree(copy_data.data); + + /* End the COPY operation */ + if (PQputCopyEnd(fmstate->conn, NULL) < 0 || PQflush(fmstate->conn)) + pgfdw_report_error(NULL, fmstate->conn, fmstate->query); + + /* + * Get the result, and check for success. + */ + res = pgfdw_get_result(fmstate->conn); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + pgfdw_report_error(res, fmstate->conn, fmstate->query); + + n_rows = atoi(PQcmdTuples(res)); + + /* And clean up */ + PQclear(res); + + reset_transmission_modes(nestlevel); + + MemoryContextReset(fmstate->temp_cxt); + + *numSlots = n_rows; + + /* + * Return NULL if nothing was inserted on the remote end + */ + return (n_rows > 0) ? slots : NULL; +} + +/* + * Write target attribute values from fmstate into buf buffer to be sent as + * COPY FROM STDIN data + */ +static void +convert_slot_to_copy_text(StringInfo buf, + PgFdwModifyState *fmstate, + TupleTableSlot *slot) +{ + TupleDesc tupdesc = RelationGetDescr(fmstate->rel); + bool first = true; + int i = 0; + + foreach_int(attnum, fmstate->target_attrs) + { + CompactAttribute *attr = TupleDescCompactAttr(tupdesc, attnum - 1); + Datum datum; + bool isnull; + + /* Ignore generated columns; they are set to DEFAULT */ + if (attr->attgenerated) + continue; + + if (!first) + appendStringInfoCharMacro(buf, '\t'); + first = false; + + datum = slot_getattr(slot, attnum, &isnull); + + if (isnull) + appendStringInfoString(buf, "\\N"); + else + { + const char *value = OutputFunctionCall(&fmstate->p_flinfo[i], + datum); + + /* Escape the value if needed */ + appendStringInfoText(buf, value); + } + i++; + } + + appendStringInfoCharMacro(buf, '\n'); +} + +/* Append a string to buf, escaping special characters for COPY TEXT format. */ +static void +appendStringInfoText(StringInfo buf, const char *string) +{ + CopyEscapeText(buf, + string, + '\t', + PG_UTF8, + false, + false); +} diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h index a2bb1ff352c..fc6922ddd4f 100644 --- a/contrib/postgres_fdw/postgres_fdw.h +++ b/contrib/postgres_fdw/postgres_fdw.h @@ -204,6 +204,7 @@ extern void rebuildInsertSql(StringInfo buf, Relation rel, char *orig_query, List *target_attrs, int values_end_len, int num_params, int num_rows); +extern void deparseCopySql(StringInfo buf, Relation rel, List *target_attrs); extern void deparseUpdateSql(StringInfo buf, RangeTblEntry *rte, Index rtindex, Relation rel, List *targetAttrs, diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql index dfc58beb0d2..db95b148ca8 100644 --- a/contrib/postgres_fdw/sql/postgres_fdw.sql +++ b/contrib/postgres_fdw/sql/postgres_fdw.sql @@ -1970,6 +1970,37 @@ select * from gloc1; select * from grem1; delete from grem1; +-- test that fdw also use COPY FROM as a remote sql +set client_min_messages to 'log'; + +create function insert_or_copy() returns trigger as $$ +declare query text; +begin + query := current_query(); + raise notice '%', query; +return new; +end; +$$ language plpgsql; + +CREATE TRIGGER trig_row_before +BEFORE INSERT OR UPDATE OR DELETE ON gloc1 +FOR EACH ROW EXECUTE PROCEDURE insert_or_copy(); + +copy grem1 from stdin; +3 +\. + +drop trigger trig_row_before on gloc1; +reset client_min_messages; + +-- test that copy does not fail with column_name alias +create table gloc2(xxx int); +create foreign table grem2(a int) server loopback options(table_name 'gloc2'); +alter foreign table grem2 alter column a options (column_name 'xxx'); +copy grem2 from stdin; +1 +\. + -- test batch insert alter server loopback options (add batch_size '10'); explain (verbose, costs off) @@ -1996,6 +2027,24 @@ drop table tab_batch_local; drop table tab_batch_sharded; drop table tab_batch_sharded_p1_remote; +-- test batch insert using copy +set client_min_messages to 'debug1'; +copy grem1 from stdin; +1 +2 +3 +4 +5 +6 +7 +8 +9 +10 +11 +12 +\. +reset client_min_messages; + alter server loopback options (drop batch_size); -- =================================================================== @@ -3060,6 +3109,39 @@ drop trigger trig_null on loc2; delete from rem2; +-- Test COPY FROM with column list and special characters +copy rem2 (f1, f2) from stdin; +1 hello\nworld +\. +select * from rem2; + +delete from rem2; + +-- Test COPY with NULL and special characters +copy rem2 from stdin; +1 \N +\N bar +3 a"b +\. +select * from rem2; + +delete from rem2; + +-- Test that float numbers do not loose precision when sending to the foreign +-- server +create table f(a float); +create foreign table f_fdw(a float) server loopback options(table_name 'f'); + +set extra_float_digits = 0; +copy f_fdw from stdin; +1.0000000000000002 +\. + +reset extra_float_digits; +select * from f; + +drop table f; + -- Check with zero-column foreign table; batch insert will be disabled alter table loc2 drop column f1; alter table loc2 drop column f2; -- 2.50.1 (Apple Git-155) ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: postgres_fdw: Use COPY to speed up batch inserts @ 2026-05-27 23:25 Matheus Alcantara <[email protected]> parent: Matheus Alcantara <[email protected]> 1 sibling, 0 replies; 35+ messages in thread From: Matheus Alcantara @ 2026-05-27 23:25 UTC (permalink / raw) To: solaimurugan vellaipandiyan <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Andrew Dunstan <[email protected]>; jian he <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers On 07/05/26 08:38, solaimurugan vellaipandiyan wrote: > Hi all, > > Thank you for the updated Patch. > > ... > > Looking forward to more feedback on this. > Hi, Thank you for reviewing and testing the patch. I've just posted a new version [1] removing the duplicated code. [1] https://www.postgresql.org/message-id/DITUGI3X54S8.346G3XFY7DRDK%40gmail.com -- Matheus Alcantara EDB: https://www.enterprisedb.com ^ permalink raw reply [nested|flat] 35+ messages in thread
end of thread, other threads:[~2026-05-27 23:25 UTC | newest] Thread overview: 35+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2018-09-26 10:20 64-bit hash function for hstore and citext data type amul sul <[email protected]> 2018-11-21 05:03 ` Hironobu SUZUKI <[email protected]> 2018-11-21 05:09 ` amul sul <[email protected]> 2018-11-22 18:51 ` Tomas Vondra <[email protected]> 2018-11-22 20:29 ` Andrew Gierth <[email protected]> 2018-11-23 18:45 ` Tom Lane <[email protected]> 2018-11-23 19:05 ` Tom Lane <[email protected]> 2018-11-24 11:36 ` Andrew Gierth <[email protected]> 2018-11-26 04:50 ` amul sul <[email protected]> 2019-04-22 19:45 [PATCH] Unify error messages Alvaro Herrera <[email protected]> 2019-11-04 18:50 [PATCH] Change pg_restore -f- to dump to stdout instead of to ./- Alvaro Herrera <[email protected]> 2023-06-14 17:54 [PATCH v2 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-14 17:54 [PATCH v2 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-14 17:54 [PATCH v3 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-14 17:54 [PATCH v1 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-14 17:54 [PATCH v3 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-14 17:54 [PATCH v1 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-19 20:57 [PATCH v9 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-19 20:57 [PATCH v7 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-19 20:57 [PATCH v8 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-19 20:57 [PATCH v7 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-19 20:57 [PATCH v8 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-19 20:57 [PATCH v4 2/3] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-19 20:57 [PATCH v9 1/2] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2023-06-19 20:57 [PATCH v4 2/3] partial revert of ff9618e82a Nathan Bossart <[email protected]> 2026-01-02 20:15 Re: postgres_fdw: Use COPY to speed up batch inserts Masahiko Sawada <[email protected]> 2026-01-02 20:33 ` Re: postgres_fdw: Use COPY to speed up batch inserts Matheus Alcantara <[email protected]> 2026-01-03 12:45 ` Re: Re: postgres_fdw: Use COPY to speed up batch inserts Dewei Dai <[email protected]> 2026-01-05 13:43 ` Re: postgres_fdw: Use COPY to speed up batch inserts Matheus Alcantara <[email protected]> 2026-01-27 19:17 ` Re: postgres_fdw: Use COPY to speed up batch inserts Masahiko Sawada <[email protected]> 2026-01-29 14:02 ` Re: postgres_fdw: Use COPY to speed up batch inserts Matheus Alcantara <[email protected]> 2026-02-26 01:39 ` Re: postgres_fdw: Use COPY to speed up batch inserts Masahiko Sawada <[email protected]> 2026-02-26 15:58 ` Re: postgres_fdw: Use COPY to speed up batch inserts Matheus Alcantara <[email protected]> 2026-05-27 23:22 ` Re: postgres_fdw: Use COPY to speed up batch inserts Matheus Alcantara <[email protected]> 2026-05-27 23:25 ` Re: postgres_fdw: Use COPY to speed up batch inserts Matheus Alcantara <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox