public inbox for [email protected]help / color / mirror / Atom feed
[PATCH 8/9] Define multi-minmax oclasses for types without distance 2+ messages / 2 participants [nested] [flat]
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 2+ messages in thread
* Clean up find_typedefs and add support for Mac @ 2023-12-12 21:16 Tristan Partin <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Tristan Partin @ 2023-12-12 21:16 UTC (permalink / raw) To: pgsql-hackers The script was using a few deprecated things according to POSIX: - -o instead of || - egrep - `` instead of $() I removed those for their "modern" equivalents. Hopefully no buildfarm member complains. I can remove any of those patches though. I did go ahead and remove egrep usage from the entire codebase while I was at it. There is still a configure check though. I'm thinking that could also be removed? I moved system detection to use uname -s. I hope that isn't a big deal. Not sure the best way to identify Mac otherwise. The big patch here is adding support for Mac. objdump -W doesn't work on Mac. So, I used dsymutil and dwarfdump to achieve the same result. I am not someone who ever uses awk, so someone should definitely check my work there. I can only confirm this works on the latest version of Mac, and have no clue how backward compatible it is. I also wrote this without having a Mac. I had to ping a coworker with a Mac for help. My goal with the Mac support is to enable use of find_typedef for extension developers, where using a Mac might be more prominent than upstream Postgres development, but that is just a guess. -- Tristan Partin Neon (https://neon.tech) Attachments: [text/x-patch] v1-0001-Replace-egrep-with-grep-E.patch (4.1K, ../../[email protected]/2-v1-0001-Replace-egrep-with-grep-E.patch) download | inline diff: From e717a14a171c0226921ffed003dedd104bf3cf99 Mon Sep 17 00:00:00 2001 From: Tristan Partin <[email protected]> Date: Tue, 12 Dec 2023 10:13:13 -0600 Subject: [PATCH v1 1/6] Replace egrep with grep -E GNU grep has been warning about grep -E since the 3.8 series. Further GNU commentary: > 7th Edition Unix had commands egrep and fgrep that were the counterparts > of the modern grep -E and grep -F. Although breaking up grep into three > programs was perhaps useful on the small computers of the 1970s, egrep > and fgrep were not standardized by POSIX and are no longer needed. In > the current GNU implementation, egrep and fgrep issue a warning and then > act like their modern counterparts; eventually, they are planned to be > removed entirely. Further man page documentation: > This grep has been enhanced in an upwards-compatible way to provide the > exact functionality of the historical egrep and fgrep commands as well. > It was the clear intention of the standard developers to consolidate the > three greps into a single command. --- src/backend/port/aix/mkldexport.sh | 4 ++-- src/tools/find_typedef | 6 +++--- src/tools/perlcheck/find_perl_files | 2 +- src/tools/pginclude/pgrminclude | 6 +++--- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/src/backend/port/aix/mkldexport.sh b/src/backend/port/aix/mkldexport.sh index adf3793e86..41405eba02 100755 --- a/src/backend/port/aix/mkldexport.sh +++ b/src/backend/port/aix/mkldexport.sh @@ -53,9 +53,9 @@ else fi fi $NM -BCg $1 | \ - egrep ' [TDB] ' | \ + grep -E ' [TDB] ' | \ sed -e 's/.* //' | \ - egrep -v '\$' | \ + grep -E -v '\$' | \ sed -e 's/^[.]//' | \ sort | \ uniq diff --git a/src/tools/find_typedef b/src/tools/find_typedef index 24e9b76651..fec0520c32 100755 --- a/src/tools/find_typedef +++ b/src/tools/find_typedef @@ -36,12 +36,12 @@ do # if objdump -W is recognized, only one line of error should appear if [ `objdump -W 2>&1 | wc -l` -eq 1 ] then # Linux objdump -W "$DIR"/* | - egrep -A3 '\(DW_TAG_typedef\)' | + grep -E -A3 '\(DW_TAG_typedef\)' | awk ' $2 == "DW_AT_name" {print $NF}' elif [ `readelf -w 2>&1 | wc -l` -gt 1 ] then # FreeBSD, similar output to Linux readelf -w "$DIR"/* | - egrep -A3 '\(DW_TAG_typedef\)' | + grep -E -A3 '\(DW_TAG_typedef\)' | awk ' $1 == "DW_AT_name" {print $NF}' fi done | @@ -50,4 +50,4 @@ sort | uniq | # these are used both for typedefs and variable names # so do not include them -egrep -v '^(date|interval|timestamp|ANY)$' +grep -E -v '^(date|interval|timestamp|ANY)$' diff --git a/src/tools/perlcheck/find_perl_files b/src/tools/perlcheck/find_perl_files index 20dceb800d..406ec7f3a0 100644 --- a/src/tools/perlcheck/find_perl_files +++ b/src/tools/perlcheck/find_perl_files @@ -12,7 +12,7 @@ find_perl_files () { find "$@" -type f -name '*.p[lm]' -print # take executable files that file(1) thinks are perl files find "$@" -type f -perm -100 -exec file {} \; -print | - egrep -i ':.*perl[0-9]*\>' | + grep -E -i ':.*perl[0-9]*\>' | cut -d: -f1 } | sort -u | grep -v '^\./\.git/' } diff --git a/src/tools/pginclude/pgrminclude b/src/tools/pginclude/pgrminclude index 7cbd2e7c9c..27c6c5bfb5 100755 --- a/src/tools/pginclude/pgrminclude +++ b/src/tools/pginclude/pgrminclude @@ -64,14 +64,14 @@ compile_file() { [ "$INCLUDE" = "pg_config.h" ] && continue [ "$INCLUDE" = "c.h" ] && continue # Stringify macros will expand undefined identifiers, so skip files that use it - egrep -q '\<(CppAsString2?|CppConcat)\>' "$FILE" && continue + grep -E -q '\<(CppAsString2?|CppConcat)\>' "$FILE" && continue # preserve configure-specific includes # these includes are surrounded by #ifdef's grep -B1 '^#include[ ][ ]*[<"]'"$INCLUDE"'[>"]' "$FILE" | - egrep -q '^#if|^#else|^#elif' && continue + grep -E -q '^#if|^#else|^#elif' && continue grep -A1 '^#include[ ][ ]*[<"]'"$INCLUDE"'[>"]' "$FILE" | - egrep -q '^#else|^#elif|^#endif' && continue + grep -E -q '^#else|^#elif|^#endif' && continue # Remove all #if and #ifdef blocks because the blocks # might contain code that is not compiled on this platform. -- Tristan Partin Neon (https://neon.tech) [text/x-patch] v1-0002-Remove-use-of-test-o-in-find_typedef.patch (1.5K, ../../[email protected]/3-v1-0002-Remove-use-of-test-o-in-find_typedef.patch) download | inline diff: From 59065b269c86747ece904b4d7a4bd9f494476978 Mon Sep 17 00:00:00 2001 From: Tristan Partin <[email protected]> Date: Tue, 12 Dec 2023 10:21:12 -0600 Subject: [PATCH v1 2/6] Remove use of test -o in find_typedef This has been deprecated for a while. Rationale from POSIX: > The XSI extensions specifying the -a and -o binary primaries and the '(' > and ')' operators have been marked obsolescent. (Many expressions using > them are ambiguously defined by the grammar depending on the specific > expressions being evaluated.) Scripts using these expressions should be > converted to the forms given below. Even though many implementations > will continue to support these obsolescent forms, scripts should be > extremely careful when dealing with user-supplied input that could be > confused with these and other primaries and operators. Unless the > application developer knows all the cases that produce input to the > script, invocations like: > > test "$1" -a "$2" > > should be written as: > > test "$1" && test "$2" --- src/tools/find_typedef | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tools/find_typedef b/src/tools/find_typedef index fec0520c32..ac798a90c1 100755 --- a/src/tools/find_typedef +++ b/src/tools/find_typedef @@ -26,7 +26,7 @@ # http://www.informatik.uni-frankfurt.de/doc/texi/stabs_toc.html -if [ "$#" -eq 0 -o ! -d "$1" ] +if [ "$#" -eq 0 ] || [ ! -d "$1" ] then echo "Usage: $0 postgres_binary_directory [...]" 1>&2 exit 1 fi -- Tristan Partin Neon (https://neon.tech) [text/x-patch] v1-0003-Remove-use-of-backticks-for-running-processes-in-.patch (1.3K, ../../[email protected]/4-v1-0003-Remove-use-of-backticks-for-running-processes-in-.patch) download | inline diff: From 23917026c656af38ef6d781622184d06dcf72ed3 Mon Sep 17 00:00:00 2001 From: Tristan Partin <[email protected]> Date: Tue, 12 Dec 2023 10:25:49 -0600 Subject: [PATCH v1 3/6] Remove use of backticks for running processes in find_typedef Backticks are the legacy form of command substitution, required by only the very oldest of non-POSIX-compatible Bourne shells. Per POSIX: > The "$()" form of command substitution solves a problem of inconsistent > behavior when using backquotes. Link: https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xcu_chap02.html#tag_23_02_06_03 --- src/tools/find_typedef | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/tools/find_typedef b/src/tools/find_typedef index ac798a90c1..e04c642da8 100755 --- a/src/tools/find_typedef +++ b/src/tools/find_typedef @@ -33,12 +33,12 @@ fi for DIR do # if objdump -W is recognized, only one line of error should appear - if [ `objdump -W 2>&1 | wc -l` -eq 1 ] + if [ $(objdump -W 2>&1 | wc -l) -eq 1 ] then # Linux objdump -W "$DIR"/* | grep -E -A3 '\(DW_TAG_typedef\)' | awk ' $2 == "DW_AT_name" {print $NF}' - elif [ `readelf -w 2>&1 | wc -l` -gt 1 ] + elif [ $(readelf -w 2>&1 | wc -l) -gt 1 ] then # FreeBSD, similar output to Linux readelf -w "$DIR"/* | grep -E -A3 '\(DW_TAG_typedef\)' | -- Tristan Partin Neon (https://neon.tech) [text/x-patch] v1-0004-Use-uname-s-for-determining-the-system-in-find_ty.patch (1.4K, ../../[email protected]/5-v1-0004-Use-uname-s-for-determining-the-system-in-find_ty.patch) download | inline diff: From ec28f5a3f7d9c8854dcdd02f57d1f892f51cb7c0 Mon Sep 17 00:00:00 2001 From: Tristan Partin <[email protected]> Date: Tue, 12 Dec 2023 13:22:19 -0600 Subject: [PATCH v1 4/6] Use uname -s for determining the system in find_typedef Analyzing error output has proven to be unstable per a patch from Tom after discovering we were missing some typedefs. uname(1) is standardized by POSIX, and can be relied upon to be much more stable. Link: https://www.postgresql.org/message-id/flat/526703.1652385613%40sss.pgh.pa.us --- src/tools/find_typedef | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/src/tools/find_typedef b/src/tools/find_typedef index e04c642da8..8adbe9ab94 100755 --- a/src/tools/find_typedef +++ b/src/tools/find_typedef @@ -31,15 +31,17 @@ then echo "Usage: $0 postgres_binary_directory [...]" 1>&2 exit 1 fi +system=$(uname -s) + for DIR -do # if objdump -W is recognized, only one line of error should appear - if [ $(objdump -W 2>&1 | wc -l) -eq 1 ] - then # Linux +do + if [ "$system" = Linux ] + then objdump -W "$DIR"/* | grep -E -A3 '\(DW_TAG_typedef\)' | awk ' $2 == "DW_AT_name" {print $NF}' - elif [ $(readelf -w 2>&1 | wc -l) -gt 1 ] - then # FreeBSD, similar output to Linux + elif [ "$system" = FreeBSD ] + then readelf -w "$DIR"/* | grep -E -A3 '\(DW_TAG_typedef\)' | awk ' $1 == "DW_AT_name" {print $NF}' -- Tristan Partin Neon (https://neon.tech) [text/x-patch] v1-0005-Add-Mac-support-to-find_typedef.patch (1.0K, ../../[email protected]/6-v1-0005-Add-Mac-support-to-find_typedef.patch) download | inline diff: From ed8a89fca03e264ad772f738bc966bbbe0d31b4d Mon Sep 17 00:00:00 2001 From: Tristan Partin <[email protected]> Date: Tue, 12 Dec 2023 13:43:32 -0600 Subject: [PATCH v1 5/6] Add Mac support to find_typedef This was previously unsupported on Mac. --- src/tools/find_typedef | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/tools/find_typedef b/src/tools/find_typedef index 8adbe9ab94..ce82897c3f 100755 --- a/src/tools/find_typedef +++ b/src/tools/find_typedef @@ -45,6 +45,17 @@ do readelf -w "$DIR"/* | grep -E -A3 '\(DW_TAG_typedef\)' | awk ' $1 == "DW_AT_name" {print $NF}' + elif [ "$system" = Darwin ] + then + # Creates *.dwarf files that we will remove later + dsymutil --flat "$DIR"/* + + # Apple's dwarfdump, wraps types: ("example") + dwarfdump "$DIR"/*.dwarf | + grep -F -A3 'DW_TAG_typedef' | + awk ' $1 == "DW_AT_name" {print substr($NF, 3, length($NF) - 4)}' + + rm -f "$DIR"/*.dwarf fi done | grep -v ' ' | # some typedefs have spaces, remove them -- Tristan Partin Neon (https://neon.tech) [text/x-patch] v1-0006-Use-grep-F-instead-of-E-in-find_typedef.patch (945B, ../../[email protected]/7-v1-0006-Use-grep-F-instead-of-E-in-find_typedef.patch) download | inline diff: From b741122e2d4fe3c33cd1ad7c170f54a2c7e2c2f1 Mon Sep 17 00:00:00 2001 From: Tristan Partin <[email protected]> Date: Tue, 12 Dec 2023 14:37:32 -0600 Subject: [PATCH v1 6/6] Use grep -F instead of -E in find_typedef These grep don't need regex support. --- src/tools/find_typedef | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/tools/find_typedef b/src/tools/find_typedef index ce82897c3f..04f6d45021 100755 --- a/src/tools/find_typedef +++ b/src/tools/find_typedef @@ -38,12 +38,12 @@ do if [ "$system" = Linux ] then objdump -W "$DIR"/* | - grep -E -A3 '\(DW_TAG_typedef\)' | + grep -F -A3 DW_TAG_typedef | awk ' $2 == "DW_AT_name" {print $NF}' elif [ "$system" = FreeBSD ] then readelf -w "$DIR"/* | - grep -E -A3 '\(DW_TAG_typedef\)' | + grep -F -A3 DW_TAG_typedef | awk ' $1 == "DW_AT_name" {print $NF}' elif [ "$system" = Darwin ] then -- Tristan Partin Neon (https://neon.tech) ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2023-12-12 21:16 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2023-12-12 21:16 Clean up find_typedefs and add support for Mac Tristan Partin <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox