public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 8/9] Define multi-minmax oclasses for types without distance
2+ messages / 2 participants
[nested] [flat]
* [PATCH 8/9] Define multi-minmax oclasses for types without distance
@ 2021-02-03 18:12 Tomas Vondra <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Tomas Vondra @ 2021-02-03 18:12 UTC (permalink / raw)
The existing multi-minmax opclasses rely on "distance" function when
deciding how to build the ranges, covering all the values. In principle,
those opclasses try to minimize the lengths of ranges, i.e. maximize the
lengths of "gaps" between them.
For some data types it's hard to construct a distance function - types
like "text" or "name" generally serve as labels, and may have ordering
only. So this uses a different approach, based on the observation that
it's the outliers that "break" BRIN minmax indexes, i.e. the lowest and
highest values. So simply sort the values, keep the (K-2) extreme values
on either tail, and build a single range representing the values in the
middle. Of course, the question is whether this is actually useful, but
that's hard to judge, and it's the best we can do.
---
src/backend/access/brin/brin_minmax_multi.c | 75 ++++++++++++
src/include/catalog/pg_amop.dat | 119 ++++++++++++++++++++
src/include/catalog/pg_amproc.dat | 113 +++++++++++++++++++
src/include/catalog/pg_opclass.dat | 21 ++++
src/include/catalog/pg_opfamily.dat | 14 +++
5 files changed, 342 insertions(+)
diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c
index 08d0d55b06..d11ec73c81 100644
--- a/src/backend/access/brin/brin_minmax_multi.c
+++ b/src/backend/access/brin/brin_minmax_multi.c
@@ -1210,6 +1210,9 @@ build_distances(FmgrInfo *distanceFn, Oid colloid,
Assert(ncranges >= 2);
+ if (!distanceFn)
+ return NULL;
+
distances = (DistanceValue *) palloc0(sizeof(DistanceValue) * (ncranges - 1));
/*
@@ -1298,6 +1301,70 @@ count_values(CombineRange *cranges, int ncranges)
}
#endif
+
+static int
+reduce_combine_ranges_simple(CombineRange *cranges, int ncranges,
+ int max_values, FmgrInfo *cmp, Oid colloid)
+{
+ int i;
+ int nvalues;
+ Datum *values;
+
+ compare_context cxt;
+
+ /* number of values to keep on each tail */
+ int tail = (max_values - 2) / 2;
+ int m = Min(tail / 2, ncranges - 1 / 2);
+
+ /* allocate space for the boundary values */
+ nvalues = 0;
+ values = (Datum *) palloc(sizeof(Datum) * max_values);
+
+ for (i = 0; i < m; i++)
+ {
+ /* head */
+ values[nvalues++] = cranges[i].minval;
+ values[nvalues++] = cranges[i].maxval;
+
+ /* tail */
+ values[nvalues++] = cranges[ncranges - 1 - i].minval;
+ values[nvalues++] = cranges[ncranges - 1 - i].maxval;
+ }
+
+ /* middle part */
+ values[nvalues++] = cranges[m].maxval;
+ values[nvalues++] = cranges[ncranges - 1 - m].minval;
+
+ /* We should have even number of range values. */
+ Assert(nvalues % 2 == 0);
+
+ /* sort the values */
+ cxt.colloid = colloid;
+ cxt.cmpFn = cmp;
+
+ /*
+ * Sort the values using the comparator function, and form ranges
+ * from the sorted result.
+ */
+ qsort_arg(values, nvalues, sizeof(Datum),
+ compare_values, (void *) &cxt);
+
+ /* We have nvalues boundary values, which means nvalues/2 ranges. */
+ for (i = 0; i < (nvalues / 2); i++)
+ {
+ cranges[i].minval = values[2*i];
+ cranges[i].maxval = values[2*i + 1];
+
+ /* if the boundary values are the same, it's a collapsed range */
+ cranges[i].collapsed = (compare_values(&values[2*i],
+ &values[2*i+1],
+ &cxt) == 0);
+ }
+
+ return (nvalues / 2);
+}
+
+
/*
* reduce_combine_ranges
* reduce the ranges until the number of values is low enough
@@ -1366,6 +1433,14 @@ reduce_combine_ranges(CombineRange *cranges, int ncranges,
if (keep >= ndistances)
return ncranges;
+ /*
+ * Without distances, we use a simple approach keeping as many
+ * outliers as possible.
+ */
+ if (!distances)
+ return reduce_combine_ranges_simple(cranges, ncranges, max_values,
+ cmp, colloid);
+
/* sort the values */
cxt.colloid = colloid;
cxt.cmpFn = cmp;
diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat
index 8135854163..5cf820f04b 100644
--- a/src/include/catalog/pg_amop.dat
+++ b/src/include/catalog/pg_amop.dat
@@ -1814,6 +1814,23 @@
amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)',
amopmethod => 'brin' },
+# minmax multi bytea
+{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea',
+ amoprighttype => 'bytea', amopstrategy => '1', amopopr => '<(bytea,bytea)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea',
+ amoprighttype => 'bytea', amopstrategy => '2', amopopr => '<=(bytea,bytea)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea',
+ amoprighttype => 'bytea', amopstrategy => '3', amopopr => '=(bytea,bytea)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea',
+ amoprighttype => 'bytea', amopstrategy => '4', amopopr => '>=(bytea,bytea)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bytea_minmax_multi_ops', amoplefttype => 'bytea',
+ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)',
+ amopmethod => 'brin' },
+
# bloom bytea
{ amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea',
amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)',
@@ -1836,6 +1853,23 @@
amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)',
amopmethod => 'brin' },
+# minmax multi "char"
+{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char',
+ amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char',
+ amoprighttype => 'char', amopstrategy => '2', amopopr => '<=(char,char)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char',
+ amoprighttype => 'char', amopstrategy => '3', amopopr => '=(char,char)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char',
+ amoprighttype => 'char', amopstrategy => '4', amopopr => '>=(char,char)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/char_minmax_multi_ops', amoplefttype => 'char',
+ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)',
+ amopmethod => 'brin' },
+
# bloom "char"
{ amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char',
amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)',
@@ -1858,6 +1892,23 @@
amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)',
amopmethod => 'brin' },
+# minmax multi name
+{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name',
+ amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name',
+ amoprighttype => 'name', amopstrategy => '2', amopopr => '<=(name,name)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name',
+ amoprighttype => 'name', amopstrategy => '3', amopopr => '=(name,name)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name',
+ amoprighttype => 'name', amopstrategy => '4', amopopr => '>=(name,name)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/name_minmax_multi_ops', amoplefttype => 'name',
+ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)',
+ amopmethod => 'brin' },
+
# bloom name
{ amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name',
amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)',
@@ -2186,6 +2237,23 @@
amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)',
amopmethod => 'brin' },
+# minmax multi text
+{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text',
+ amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text',
+ amoprighttype => 'text', amopstrategy => '2', amopopr => '<=(text,text)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text',
+ amoprighttype => 'text', amopstrategy => '3', amopopr => '=(text,text)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text',
+ amoprighttype => 'text', amopstrategy => '4', amopopr => '>=(text,text)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/text_minmax_multi_ops', amoplefttype => 'text',
+ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)',
+ amopmethod => 'brin' },
+
# bloom text
{ amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text',
amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)',
@@ -2562,6 +2630,23 @@
amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)',
amopmethod => 'brin' },
+# minmax multi character
+{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar',
+ amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '<(bpchar,bpchar)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar',
+ amoprighttype => 'bpchar', amopstrategy => '2',
+ amopopr => '<=(bpchar,bpchar)', amopmethod => 'brin' },
+{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar',
+ amoprighttype => 'bpchar', amopstrategy => '3', amopopr => '=(bpchar,bpchar)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar',
+ amoprighttype => 'bpchar', amopstrategy => '4',
+ amopopr => '>=(bpchar,bpchar)', amopmethod => 'brin' },
+{ amopfamily => 'brin/bpchar_minmax_multi_ops', amoplefttype => 'bpchar',
+ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)',
+ amopmethod => 'brin' },
+
# bloom character
{ amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar',
amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)',
@@ -3007,6 +3092,23 @@
amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)',
amopmethod => 'brin' },
+# minmax multi bit
+{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit',
+ amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit',
+ amoprighttype => 'bit', amopstrategy => '2', amopopr => '<=(bit,bit)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit',
+ amoprighttype => 'bit', amopstrategy => '3', amopopr => '=(bit,bit)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit',
+ amoprighttype => 'bit', amopstrategy => '4', amopopr => '>=(bit,bit)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/bit_minmax_multi_ops', amoplefttype => 'bit',
+ amoprighttype => 'bit', amopstrategy => '5', amopopr => '>(bit,bit)',
+ amopmethod => 'brin' },
+
# minmax bit varying
{ amopfamily => 'brin/varbit_minmax_ops', amoplefttype => 'varbit',
amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)',
@@ -3024,6 +3126,23 @@
amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)',
amopmethod => 'brin' },
+# minmax multi bit varying
+{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit',
+ amoprighttype => 'varbit', amopstrategy => '1', amopopr => '<(varbit,varbit)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit',
+ amoprighttype => 'varbit', amopstrategy => '2',
+ amopopr => '<=(varbit,varbit)', amopmethod => 'brin' },
+{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit',
+ amoprighttype => 'varbit', amopstrategy => '3', amopopr => '=(varbit,varbit)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit',
+ amoprighttype => 'varbit', amopstrategy => '4',
+ amopopr => '>=(varbit,varbit)', amopmethod => 'brin' },
+{ amopfamily => 'brin/varbit_minmax_multi_ops', amoplefttype => 'varbit',
+ amoprighttype => 'varbit', amopstrategy => '5', amopopr => '>(varbit,varbit)',
+ amopmethod => 'brin' },
+
# minmax numeric
{ amopfamily => 'brin/numeric_minmax_ops', amoplefttype => 'numeric',
amoprighttype => 'numeric', amopstrategy => '1',
diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat
index 51403716b1..69a7ed682c 100644
--- a/src/include/catalog/pg_amproc.dat
+++ b/src/include/catalog/pg_amproc.dat
@@ -805,6 +805,22 @@
{ amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea',
amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' },
+# minmax multi bytea
+{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '1',
+ amproc => 'brin_minmax_multi_opcinfo' },
+{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '2',
+ amproc => 'brin_minmax_multi_add_value' },
+{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '3',
+ amproc => 'brin_minmax_multi_consistent' },
+{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_multi_union' },
+{ amprocfamily => 'brin/bytea_minmax_multi_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '5',
+ amproc => 'brin_minmax_multi_options' },
+
# bloom bytea
{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea',
amprocrighttype => 'bytea', amprocnum => '1',
@@ -836,6 +852,22 @@
{ amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char',
amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' },
+# minmax multi "char"
+{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '1',
+ amproc => 'brin_minmax_multi_opcinfo' },
+{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '2',
+ amproc => 'brin_minmax_multi_add_value' },
+{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '3',
+ amproc => 'brin_minmax_multi_consistent' },
+{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_multi_union' },
+{ amprocfamily => 'brin/char_minmax_multi_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '5',
+ amproc => 'brin_minmax_multi_options' },
+
# bloom "char"
{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char',
amprocrighttype => 'char', amprocnum => '1',
@@ -867,6 +899,22 @@
{ amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name',
amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' },
+# minmax multi name
+{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '1',
+ amproc => 'brin_minmax_multi_opcinfo' },
+{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '2',
+ amproc => 'brin_minmax_multi_add_value' },
+{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '3',
+ amproc => 'brin_minmax_multi_consistent' },
+{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_multi_union' },
+{ amprocfamily => 'brin/name_minmax_multi_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '5',
+ amproc => 'brin_minmax_multi_options' },
+
# bloom name
{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name',
amprocrighttype => 'name', amprocnum => '1',
@@ -1197,6 +1245,22 @@
{ amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text',
amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' },
+# minmax multi text
+{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '1',
+ amproc => 'brin_minmax_multi_opcinfo' },
+{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '2',
+ amproc => 'brin_minmax_multi_add_value' },
+{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '3',
+ amproc => 'brin_minmax_multi_consistent' },
+{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_multi_union' },
+{ amprocfamily => 'brin/text_minmax_multi_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '5',
+ amproc => 'brin_minmax_multi_options' },
+
# bloom text
{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text',
amprocrighttype => 'text', amprocnum => '1',
@@ -1663,6 +1727,23 @@
amprocrighttype => 'bpchar', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# minmax multi character
+{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '1',
+ amproc => 'brin_minmax_multi_opcinfo' },
+{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '2',
+ amproc => 'brin_minmax_multi_add_value' },
+{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '3',
+ amproc => 'brin_minmax_multi_consistent' },
+{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '4',
+ amproc => 'brin_minmax_multi_union' },
+{ amprocfamily => 'brin/bpchar_minmax_multi_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '5',
+ amproc => 'brin_minmax_multi_options' },
+
# bloom character
{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar',
amprocrighttype => 'bpchar', amprocnum => '1',
@@ -2180,6 +2261,21 @@
{ amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit',
amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_union' },
+# minmax multi bit
+{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit',
+ amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_multi_opcinfo' },
+{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit',
+ amprocrighttype => 'bit', amprocnum => '2',
+ amproc => 'brin_minmax_multi_add_value' },
+{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit',
+ amprocrighttype => 'bit', amprocnum => '3',
+ amproc => 'brin_minmax_multi_consistent' },
+{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit',
+ amprocrighttype => 'bit', amprocnum => '4', amproc => 'brin_minmax_multi_union' },
+{ amprocfamily => 'brin/bit_minmax_multi_ops', amproclefttype => 'bit',
+ amprocrighttype => 'bit', amprocnum => '5',
+ amproc => 'brin_minmax_multi_options' },
+
# minmax bit varying
{ amprocfamily => 'brin/varbit_minmax_ops', amproclefttype => 'varbit',
amprocrighttype => 'varbit', amprocnum => '1',
@@ -2194,6 +2290,23 @@
amprocrighttype => 'varbit', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# minmax multi bit varying
+{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit',
+ amprocrighttype => 'varbit', amprocnum => '1',
+ amproc => 'brin_minmax_multi_opcinfo' },
+{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit',
+ amprocrighttype => 'varbit', amprocnum => '2',
+ amproc => 'brin_minmax_multi_add_value' },
+{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit',
+ amprocrighttype => 'varbit', amprocnum => '3',
+ amproc => 'brin_minmax_multi_consistent' },
+{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit',
+ amprocrighttype => 'varbit', amprocnum => '4',
+ amproc => 'brin_minmax_multi_union' },
+{ amprocfamily => 'brin/varbit_minmax_multi_ops', amproclefttype => 'varbit',
+ amprocrighttype => 'varbit', amprocnum => '5',
+ amproc => 'brin_minmax_multi_options' },
+
# minmax numeric
{ amprocfamily => 'brin/numeric_minmax_ops', amproclefttype => 'numeric',
amprocrighttype => 'numeric', amprocnum => '1',
diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat
index da25befefe..823f1b01fe 100644
--- a/src/include/catalog/pg_opclass.dat
+++ b/src/include/catalog/pg_opclass.dat
@@ -266,18 +266,27 @@
{ opcmethod => 'brin', opcname => 'bytea_minmax_ops',
opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea',
opckeytype => 'bytea' },
+{ opcmethod => 'brin', opcname => 'bytea_minmax_multi_ops',
+ opcfamily => 'brin/bytea_minmax_multi_ops', opcintype => 'bytea',
+ opckeytype => 'bytea', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'bytea_bloom_ops',
opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea',
opckeytype => 'bytea', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'char_minmax_ops',
opcfamily => 'brin/char_minmax_ops', opcintype => 'char',
opckeytype => 'char' },
+{ opcmethod => 'brin', opcname => 'char_minmax_multi_ops',
+ opcfamily => 'brin/char_minmax_multi_ops', opcintype => 'char',
+ opckeytype => 'char', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'char_bloom_ops',
opcfamily => 'brin/char_bloom_ops', opcintype => 'char',
opckeytype => 'char', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'name_minmax_ops',
opcfamily => 'brin/name_minmax_ops', opcintype => 'name',
opckeytype => 'name' },
+{ opcmethod => 'brin', opcname => 'name_minmax_multi_ops',
+ opcfamily => 'brin/name_minmax_multi_ops', opcintype => 'name',
+ opckeytype => 'name', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'name_bloom_ops',
opcfamily => 'brin/name_bloom_ops', opcintype => 'name',
opckeytype => 'name', opcdefault => 'f' },
@@ -311,6 +320,9 @@
{ opcmethod => 'brin', opcname => 'text_minmax_ops',
opcfamily => 'brin/text_minmax_ops', opcintype => 'text',
opckeytype => 'text' },
+{ opcmethod => 'brin', opcname => 'text_minmax_multi_ops',
+ opcfamily => 'brin/text_minmax_multi_ops', opcintype => 'text',
+ opckeytype => 'text', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'text_bloom_ops',
opcfamily => 'brin/text_bloom_ops', opcintype => 'text',
opckeytype => 'text', opcdefault => 'f' },
@@ -381,6 +393,9 @@
{ opcmethod => 'brin', opcname => 'bpchar_minmax_ops',
opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar',
opckeytype => 'bpchar' },
+{ opcmethod => 'brin', opcname => 'bpchar_minmax_multi_ops',
+ opcfamily => 'brin/bpchar_minmax_multi_ops', opcintype => 'bpchar',
+ opckeytype => 'bpchar', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'bpchar_bloom_ops',
opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar',
opckeytype => 'bpchar', opcdefault => 'f' },
@@ -440,9 +455,15 @@
opckeytype => 'timetz', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'bit_minmax_ops',
opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' },
+{ opcmethod => 'brin', opcname => 'bit_minmax_multi_ops',
+ opcfamily => 'brin/bit_minmax_multi_ops', opcintype => 'bit',
+ opckeytype => 'bit', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'varbit_minmax_ops',
opcfamily => 'brin/varbit_minmax_ops', opcintype => 'varbit',
opckeytype => 'varbit' },
+{ opcmethod => 'brin', opcname => 'varbit_minmax_multi_ops',
+ opcfamily => 'brin/varbit_minmax_multi_ops', opcintype => 'varbit',
+ opckeytype => 'varbit', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'numeric_minmax_ops',
opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric',
opckeytype => 'numeric' },
diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat
index ba9231ac8c..ffb20e72ab 100644
--- a/src/include/catalog/pg_opfamily.dat
+++ b/src/include/catalog/pg_opfamily.dat
@@ -192,6 +192,8 @@
opfmethod => 'brin', opfname => 'numeric_minmax_multi_ops' },
{ oid => '4056',
opfmethod => 'brin', opfname => 'text_minmax_ops' },
+{ oid => '9967',
+ opfmethod => 'brin', opfname => 'text_minmax_multi_ops' },
{ oid => '9902',
opfmethod => 'brin', opfname => 'text_bloom_ops' },
{ oid => '9903',
@@ -210,14 +212,20 @@
opfmethod => 'brin', opfname => 'datetime_bloom_ops' },
{ oid => '4062',
opfmethod => 'brin', opfname => 'char_minmax_ops' },
+{ oid => '9968',
+ opfmethod => 'brin', opfname => 'char_minmax_multi_ops' },
{ oid => '9906',
opfmethod => 'brin', opfname => 'char_bloom_ops' },
{ oid => '4064',
opfmethod => 'brin', opfname => 'bytea_minmax_ops' },
+{ oid => '9969',
+ opfmethod => 'brin', opfname => 'bytea_minmax_multi_ops' },
{ oid => '9907',
opfmethod => 'brin', opfname => 'bytea_bloom_ops' },
{ oid => '4065',
opfmethod => 'brin', opfname => 'name_minmax_ops' },
+{ oid => '9970',
+ opfmethod => 'brin', opfname => 'name_minmax_multi_ops' },
{ oid => '9908',
opfmethod => 'brin', opfname => 'name_bloom_ops' },
{ oid => '4068',
@@ -260,6 +268,8 @@
opfmethod => 'brin', opfname => 'network_bloom_ops' },
{ oid => '4076',
opfmethod => 'brin', opfname => 'bpchar_minmax_ops' },
+{ oid => '9971',
+ opfmethod => 'brin', opfname => 'bpchar_minmax_multi_ops' },
{ oid => '9915',
opfmethod => 'brin', opfname => 'bpchar_bloom_ops' },
{ oid => '4077',
@@ -276,8 +286,12 @@
opfmethod => 'brin', opfname => 'interval_bloom_ops' },
{ oid => '4079',
opfmethod => 'brin', opfname => 'bit_minmax_ops' },
+{ oid => '9972',
+ opfmethod => 'brin', opfname => 'bit_minmax_multi_ops' },
{ oid => '4080',
opfmethod => 'brin', opfname => 'varbit_minmax_ops' },
+{ oid => '9973',
+ opfmethod => 'brin', opfname => 'varbit_minmax_multi_ops' },
{ oid => '4081',
opfmethod => 'brin', opfname => 'uuid_minmax_ops' },
{ oid => '9938',
--
2.26.2
--------------0E72B707603BED22B4040825
Content-Type: text/x-patch; charset=UTF-8;
name="0007-Remove-the-special-batch-mode-use-a-larger--20210211.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="0007-Remove-the-special-batch-mode-use-a-larger--20210211.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: failure in 019_replslot_limit
@ 2024-02-09 15:00 Alexander Lakhin <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Alexander Lakhin @ 2024-02-09 15:00 UTC (permalink / raw)
To: Andres Freund <[email protected]>; pgsql-hackers; Alvaro Herrera <[email protected]>; Kyotaro HORIGUCHI <[email protected]>
Hello Andres,
05.04.2023 21:48, Andres Freund wrote:
> I just saw the following failure:
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mylodon&dt=2023-04-05%2017%3A47%3A03
> after a commit of mine. The symptoms look unrelated though.
>
> [17:54:42.188](258.345s) # poll_query_until timed out executing this query:
> # SELECT wal_status FROM pg_replication_slots WHERE slot_name = 'rep3'
> # expecting this output:
> # lost
> # last actual query output:
> # unreserved
> # with stderr:
> timed out waiting for slot to be lost at /home/bf/bf-build/mylodon/HEAD/pgsql/src/test/recovery/t/019_replslot_limit.pl line 400.
>
> We're expecting "lost" but are getting "unreserved".
>
> ...
>
> ISTM that this indicates that checkpointer got stuck after signalling
> 344783.
>
> Do you see any other explanation?
>
I've managed to reproduce this issue (which still persists:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=kestrel&dt=2024-02-04%2001%3A53%3A44
) and saw that it's not checkpointer, but walsender is hanging:
[12:15:03.753](34.771s) ok 17 - have walsender pid 317885
[12:15:03.875](0.122s) ok 18 - have walreceiver pid 317884
[12:15:04.808](0.933s) ok 19 - walsender termination logged
...
Last essential messages in _primary3.log are:
2024-02-09 12:15:04.823 UTC [318036][not initialized][:0] LOG: connection received: host=[local]
2024-02-09 12:15:04.823 UTC [317885][walsender][3/0:0] FATAL: terminating connection due to administrator command
2024-02-09 12:15:04.823 UTC [317885][walsender][3/0:0] STATEMENT: START_REPLICATION SLOT "rep3" 0/700000 TIMELINE 1
(then the test just queries the slot state, there are no other messages
related to walsender)
And I see the walsender process still running (I've increased the timeout
to keep the test running and to connect to the process in question), with
the following stack trace:
#0 0x00007fe4feac3d16 in epoll_wait (epfd=5, events=0x55b279b70f38, maxevents=1, timeout=timeout@entry=-1) at
../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1 0x000055b278b9ab32 in WaitEventSetWaitBlock (set=set@entry=0x55b279b70eb8, cur_timeout=cur_timeout@entry=-1,
occurred_events=occurred_events@entry=0x7ffda5ffac90, nevents=nevents@entry=1) at latch.c:1571
#2 0x000055b278b9b6b6 in WaitEventSetWait (set=0x55b279b70eb8, timeout=timeout@entry=-1,
occurred_events=occurred_events@entry=0x7ffda5ffac90, nevents=nevents@entry=1,
wait_event_info=wait_event_info@entry=100663297) at latch.c:1517
#3 0x000055b278a3f11f in secure_write (port=0x55b279b65aa0, ptr=ptr@entry=0x55b279bfbd08, len=len@entry=21470) at
be-secure.c:296
#4 0x000055b278a460dc in internal_flush () at pqcomm.c:1356
#5 0x000055b278a461d4 in internal_putbytes (s=s@entry=0x7ffda5ffad3c "E\177", len=len@entry=1) at pqcomm.c:1302
#6 0x000055b278a46299 in socket_putmessage (msgtype=<optimized out>, s=0x55b279b363c0 "SFATAL", len=112) at pqcomm.c:1483
#7 0x000055b278a48670 in pq_endmessage (buf=buf@entry=0x7ffda5ffada0) at pqformat.c:302
#8 0x000055b278d0c82a in send_message_to_frontend (edata=edata@entry=0x55b27908e500 <errordata>) at elog.c:3590
#9 0x000055b278d0cfe2 in EmitErrorReport () at elog.c:1716
#10 0x000055b278d0d17d in errfinish (filename=filename@entry=0x55b278eaa480 "postgres.c", lineno=lineno@entry=3295,
funcname=funcname@entry=0x55b278eaaef0 <__func__.16> "ProcessInterrupts") at elog.c:551
#11 0x000055b278bc41c9 in ProcessInterrupts () at postgres.c:3295
#12 0x000055b278b6c9af in WalSndLoop (send_data=send_data@entry=0x55b278b6c346 <XLogSendPhysical>) at walsender.c:2680
#13 0x000055b278b6cef1 in StartReplication (cmd=cmd@entry=0x55b279b733e0) at walsender.c:987
#14 0x000055b278b6d865 in exec_replication_command (cmd_string=cmd_string@entry=0x55b279b39c60 "START_REPLICATION SLOT
\"rep3\" 0/700000 TIMELINE 1") at walsender.c:2039
#15 0x000055b278bc7d71 in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4649
#16 0x000055b278b2329d in BackendRun (port=port@entry=0x55b279b65aa0) at postmaster.c:4464
#17 0x000055b278b263ae in BackendStartup (port=port@entry=0x55b279b65aa0) at postmaster.c:4140
#18 0x000055b278b26539 in ServerLoop () at postmaster.c:1776
#19 0x000055b278b27ac5 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x55b279b34180) at postmaster.c:1475
#20 0x000055b278a49ab0 in main (argc=4, argv=0x55b279b34180) at main.c:198
(gdb) frame 9
(gdb) print edata->message
$3 = 0x55b279b367d0 "terminating connection due to administrator command"
So it looks like walsender tries to send the message to walreceiver, which
is paused, and thus walsender gets stuck on it.
Best regards,
Alexander
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-02-09 15:00 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-02-03 18:12 [PATCH 8/9] Define multi-minmax oclasses for types without distance Tomas Vondra <[email protected]>
2024-02-09 15:00 Re: failure in 019_replslot_limit Alexander Lakhin <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox