public inbox for [email protected]help / color / mirror / Atom feed
[PATCH 8/9] Define multi-minmax oclasses for types without distance 67+ messages / 2 participants [nested] [flat]
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index fd85c18d83..c77c207e94 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------22A4B241170149838D4D1F8F Content-Type: text/x-patch; charset=UTF-8; name="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0009-Ignore-correlation-for-new-BRIN-opclasses-20210215.patc"; filename*1="h" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 90d17e5008..02aa618b49 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 762ac3acef..4ab4aff073 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2210,6 +2261,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2592,6 +2660,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3061,6 +3146,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3078,6 +3180,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------4B194FF8F3EA3786FF9EAE1F Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210203.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH 8/9] Define multi-minmax oclasses for types without distance @ 2021-02-03 18:12 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw) The existing multi-minmax opclasses rely on "distance" function when deciding how to build the ranges, covering all the values. In principle, those opclasses try to minimize the lengths of ranges, i.e. maximize the lengths of "gaps" between them. For some data types it's hard to construct a distance function - types like "text" or "name" generally serve as labels, and may have ordering only. So this uses a different approach, based on the observation that it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and highest values. So simply sort the values, keep the (K-2) extreme values on either tail, and build a single range representing the values in the middle. Of course, the question is whether this is actually useful, but that's hard to judge, and it's the best we can do. --- src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++ src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++ src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++ src/include/catalog/pg_opclass.dat | 21 ++++ src/include/catalog/pg_opfamily.dat | 14 +++ 5 files changed, 342 insertions(+) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index 08d0d55b06..d11ec73c81 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid, Assert(ncranges >= 2); + if (!distanceFn) + return NULL; + distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1)); /* @@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges) } #endif + +static int +reduce_combine_ranges_simple(CombineRange *cranges, int ncranges, + int max_values, FmgrInfo *cmp, Oid colloid) +{ + int i; + int nvalues; + Datum *values; + + compare_context cxt; + + /* number of values to keep on each tail */ + int tail = (max_values - 2) / 2; + int m = Min(tail / 2, ncranges - 1 / 2); + + /* allocate space for the boundary values */ + nvalues = 0; + values = (Datum *) palloc(sizeof(Datum) * max_values); + + for (i = 0; i < m; i++) + { + /* head */ + values[nvalues++] = cranges[i].minval; + values[nvalues++] = cranges[i].maxval; + + /* tail */ + values[nvalues++] = cranges[ncranges - 1 - i].minval; + values[nvalues++] = cranges[ncranges - 1 - i].maxval; + } + + /* middle part */ + values[nvalues++] = cranges[m].maxval; + values[nvalues++] = cranges[ncranges - 1 - m].minval; + + /* We should have even number of range values. */ + Assert(nvalues % 2 == 0); + + /* sort the values */ + cxt.colloid = colloid; + cxt.cmpFn = cmp; + + /* + * Sort the values using the comparator function, and form ranges + * from the sorted result. + */ + qsort_arg(values, nvalues, sizeof(Datum), + compare_values, (void *) &cxt); + + /* We have nvalues boundary values, which means nvalues/2 ranges. */ + for (i = 0; i < (nvalues / 2); i++) + { + cranges[i].minval = values[2*i]; + cranges[i].maxval = values[2*i + 1]; + + /* if the boundary values are the same, it's a collapsed range */ + cranges[i].collapsed = (compare_values(&values[2*i], + &values[2*i+1], + &cxt) == 0); + } + + return (nvalues / 2); +} + + /* * reduce_combine_ranges * reduce the ranges until the number of values is low enough @@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges, if (keep >= ndistances) return ncranges; + /* + * Without distances, we use a simple approach keeping as many + * outliers as possible. + */ + if (!distances) + return reduce_combine_ranges_simple(cranges, ncranges, max_values, + cmp, colloid); + /* sort the values */ cxt.colloid = colloid; cxt.cmpFn = cmp; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 8135854163..5cf820f04b 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,23 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# minmax multi bytea +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', + amopmethod => 'brin' }, + # bloom bytea { amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', @@ -1836,6 +1853,23 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# minmax multi "char" +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', + amopmethod => 'brin' }, + # bloom "char" { amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', @@ -1858,6 +1892,23 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# minmax multi name +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', + amopmethod => 'brin' }, + # bloom name { amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', @@ -2186,6 +2237,23 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# minmax multi text +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', + amopmethod => 'brin' }, + # bloom text { amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', @@ -2562,6 +2630,23 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# minmax multi character +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '2', + amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '4', + amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' }, +{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', + amopmethod => 'brin' }, + # bloom character { amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', @@ -3007,6 +3092,23 @@ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', amopmethod => 'brin' }, +# minmax multi bit +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit', + amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)', + amopmethod => 'brin' }, + # minmax bit varying { amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit', amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', @@ -3024,6 +3126,23 @@ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', amopmethod => 'brin' }, +# minmax multi bit varying +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '2', + amopopr => '<=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '4', + amopopr => '>=(varbit,varbit)', amopmethod => 'brin' }, +{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit', + amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)', + amopmethod => 'brin' }, + # minmax numeric { amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric', amoprighttype => 'numeric', amopstrategy => '1', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 51403716b1..69a7ed682c 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,22 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bytea +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom bytea { amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '1', @@ -836,6 +852,22 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi "char" +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom "char" { amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -867,6 +899,22 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi name +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom name { amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -1197,6 +1245,22 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi text +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom text { amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -1663,6 +1727,23 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi character +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # bloom character { amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', amprocrighttype => 'bpchar', amprocnum => '1', @@ -2180,6 +2261,21 @@ { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit', + amprocrighttype => 'bit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax bit varying { amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit', amprocrighttype => 'varbit', amprocnum => '1', @@ -2194,6 +2290,23 @@ amprocrighttype => 'varbit', amprocnum => '4', amproc => 'brin_minmax_union' }, +# minmax multi bit varying +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '1', + amproc => 'brin_minmax_multi_opcinfo' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '2', + amproc => 'brin_minmax_multi_add_value' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '3', + amproc => 'brin_minmax_multi_consistent' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '4', + amproc => 'brin_minmax_multi_union' }, +{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit', + amprocrighttype => 'varbit', amprocnum => '5', + amproc => 'brin_minmax_multi_options' }, + # minmax numeric { amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric', amprocrighttype => 'numeric', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index da25befefe..823f1b01fe 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,18 +266,27 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops', + opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bytea_bloom_ops', opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops', + opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_bloom_ops', opcfamily => 'brin/char_bloom_ops', opcintype => 'char', opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops', + opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_bloom_ops', opcfamily => 'brin/name_bloom_ops', opcintype => 'name', opckeytype => 'name', opcdefault => 'f' }, @@ -311,6 +320,9 @@ { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops', + opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_bloom_ops', opcfamily => 'brin/text_bloom_ops', opcintype => 'text', opckeytype => 'text', opcdefault => 'f' }, @@ -381,6 +393,9 @@ { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops', + opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bpchar_bloom_ops', opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', opckeytype => 'bpchar', opcdefault => 'f' }, @@ -440,9 +455,15 @@ opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, +{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops', + opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit', + opckeytype => 'bit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit', opckeytype => 'varbit' }, +{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops', + opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit', + opckeytype => 'varbit', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index ba9231ac8c..ffb20e72ab 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -192,6 +192,8 @@ opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9967', + opfmethod => 'brin', opfname => 'text_minmax_multi_ops' }, { oid => '9902', opfmethod => 'brin', opfname => 'text_bloom_ops' }, { oid => '9903', @@ -210,14 +212,20 @@ opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9968', + opfmethod => 'brin', opfname => 'char_minmax_multi_ops' }, { oid => '9906', opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9969', + opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' }, { oid => '9907', opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9970', + opfmethod => 'brin', opfname => 'name_minmax_multi_ops' }, { oid => '9908', opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', @@ -260,6 +268,8 @@ opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9971', + opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' }, { oid => '9915', opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', @@ -276,8 +286,12 @@ opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, +{ oid => '9972', + opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, +{ oid => '9973', + opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, { oid => '9938', -- 2.26.2 --------------0E72B707603BED22B4040825 Content-Type: text/x-patch; charset=UTF-8; name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa"; filename*1="tch" ^ permalink raw reply [nested|flat] 67+ messages in thread
* [PATCH v1 2/3] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 67+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- 3 files changed, 2 insertions(+), 57 deletions(-) 75.3% src/backend/replication/ 24.6% src/backend/utils/activity/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index a41453530a1..266379c780a 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -553,16 +553,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1ab09655a70..c33185bd337 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1826,7 +1822,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1847,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1958,8 +1952,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1968,15 +1961,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2879,8 +2863,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2975,9 +2957,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2985,7 +2964,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -2996,21 +2974,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index bc8c43b96aa..feae2ae5f44 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* -- 2.34.1 --tjEWjIIwfNIHQLgt Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v1-0003-Add-FLUSH_MIXED-support-and-implement-it-for-RELA.patch" ^ permalink raw reply [nested|flat] 67+ messages in thread
end of thread, other threads:[~2026-01-06 11:06 UTC | newest] Thread overview: 67+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]> 2026-01-06 11:06 [PATCH v1 2/3] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox