public inbox for [email protected]help / color / mirror / Atom feed
[PATCH 8/9] Define multi-minmax oclasses for types without distance 67+ messages / 2 participants [nested] [flat]
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* Re: Import Statistics in postgres_fdw before resorting to sampling. @ 2026-01-04 02:56 Corey Huinker <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Corey Huinker @ 2026-01-04 02:56 UTC (permalink / raw) To: Etsuro Fujita <[email protected]>; +Cc: Michael Paquier <[email protected]>; Ashutosh Bapat <[email protected]>; pgsql-hackers; [email protected]; [email protected] > > > This version of the patch wouldn't fall back to the normal ANALYZE > processing anymore, so this documentation should be updated as such. > Also, as I think it's the user's responsibility to ensure the existing > statistics are up-to-date, as I said before, I think we should add a > note about that here. Also, as some users wouldn't be able to ensure > it, I'm wondering if the default should be false. > I agree, if there is no fallback, then the default should be false. When I was initially brainstorming this patch, Nathan Bossart had suggested making it the default because 1) that would be an automatic benefit to users and 2) the cost for attempting to import stats was small in comparison to a table stample, so it was worth the attempt. I still want users to get that automatic benefit, but if there is no fallback to sampling then the default only makes sense as false. > If the user wasn't able to ensure the statistics are up-to-date, I > think he/she might want to do remote ANALYZE *before* fetching the > statistics, not after trying to do so. So I think we could instead > provide this option as such. What do you think about that? Anyway, > I'd vote for leaving this for another patch, as I think it's a > nice-to-have option rather than a must-have one, as I said before. > I hadn't thought of that one. I think it has some merit, but I think we'd want the try-after case as well. So the settings would be: "off" (never remote analyze, default), "on" (always analyze before fetching), and "retry" (analyze if no stats were found). This feature feeds into the same thinking that the default setting did, which was to make this feature just do what is usually the smart thing, and do it automatically. Building it in pieces might be easier to get committed, but it takes away the dream of seamless automatic improvement, and establishes defaults that can't be changed in future versions. That dream was probably unrealistic, but I wanted to try. > > ISTM that the code is well organized overall. Here are a few comments: > Thanks! > > We don't use relallvisible and relallfrozen for foreign tables (note > that do_analyze_rel() calls vac_update_relstats() with relallvisible=0 > and relallfrozen=0 for them). Do we really need to retrieve (and > restore) them? > No, and as you stated, we wouldn't want to. The query was lifted verbatim from pg_dump, with a vague hope of moving the queries to a common library that both pg_dump and postgres_fdw could draw upon. But that no longer makes sense, so I'll fix. > +static const char *attstats_query_17 = > + "SELECT DISTINCT ON (s.attname) attname, s.null_frac, s.avg_width, " > + "s.n_distinct, s.most_common_vals, s.most_common_freqs, " > + "s.histogram_bounds, s.correlation, s.most_common_elems, " > + "s.most_common_elem_freqs, s.elem_count_histogram, " > + "s.range_length_histogram, s.range_empty_frac, > s.range_bounds_histogram " > + "FROM pg_catalog.pg_stats AS s " > + "WHERE s.schemaname = $1 AND s.tablename = $2 " > + "ORDER BY s.attname, s.inherited DESC"; > > I think we should retrieve the attribute statistics for only the > referenced columns of the remote table, not all the columns of it, to > reduce the data transfer and the cost of matching local/remote > attributes in import_fetched_statistics(). > I thought about this, and decided that we wanted to 1) avoid per-column round trips and 2) keep the remote queries simple. It's not such a big deal to add "AND s.attname = ANY($3)" and construct a '{att1,att2,"ATt3"}' string, as we already do in pg_dump in a few places. > > In fetch_remote_statistics() > > + if (server_version_num >= 180000) > + relation_sql = relstats_query_18; > + else if (server_version_num >= 140000) > + relation_sql = relstats_query_14; > + else > + relation_sql = relstats_query_default; > > I think that having the definition for each of relstats_query_18, > relstats_query_14 and relstats_query_default makes the code redundant > and the maintenance hard. To avoid that, how about building the > relation_sql query dynamically as done for the query to fetch all > table data from the remote server in postgresImportForeignSchema(). > Same for the attribute_sql query. > It may be a moot point. If we're not fetching relallfrozen, then the 14 & 18 cases are now the same, and since the pre-14 case concerns differentiating between analyzed and unanalyzed tables, we would just map that to 0 IF we kept those stats, but we almost never would because an unanalyzed remote table would not have the attribute stats necessary to qualify as a good remote fetch. So we're down to just one static query. > > That's all I have for now. I will continue to review the changes. > Much appreciated. ^ permalink raw reply [nested|flat] 67+ messages in thread
end of thread, other threads:[~2026-01-04 02:56 UTC | newest] Thread overview: 67+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2026-01-04 02:56 Re: Import Statistics in postgres_fdw before resorting to sampling. Corey Huinker <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox