public inbox for [email protected]help / color / mirror / Atom feed
[PATCH 6/8] BRIN bloom indexes 11+ messages / 6 participants [nested] [flat]
* [PATCH 6/8] BRIN bloom indexes @ 2020-12-16 20:24 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 11+ messages in thread From: Tomas Vondra @ 2020-12-16 20:24 UTC (permalink / raw) Adds a BRIN opclass using a Bloom filter to summarize the range. BRIN indexes using the new opclasses only allow equality queries, but that works for data like UUID, MAC addresses etc. for which range queries are not very common (and the data look random, making BRIN minmax indexes inefficient anyway). BRIN bloom opclasses allow specifying the usual Bloom parameters, like expected number of distinct values (in the BRIN range) and desired false positive rate. The opclasses store 32-bit hashes of the values, not the raw indexed values. This assumes the hash functions for data types has low number of collisions, good performance etc. Collisions are not a huge issue though, because the number of values in a BRIN ranges is fairly small. Author: Tomas Vondra <[email protected]> Reviewed-by: Alvaro Herrera <[email protected]> Reviewed-by: Alexander Korotkov <[email protected]> Reviewed-by: Sokolov Yura <[email protected]> Reviewed-by: John Naylor <[email protected]> Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/5d78b774-7e9c-c94e-12cf-fef51cc89b1a%402ndquadrant.com --- doc/src/sgml/brin.sgml | 230 ++++++- doc/src/sgml/ref/create_index.sgml | 1 + src/backend/access/brin/Makefile | 1 + src/backend/access/brin/brin_bloom.c | 787 ++++++++++++++++++++++ src/include/access/brin.h | 2 + src/include/access/brin_internal.h | 4 + src/include/catalog/pg_amop.dat | 116 ++++ src/include/catalog/pg_amproc.dat | 447 ++++++++++++ src/include/catalog/pg_opclass.dat | 72 ++ src/include/catalog/pg_opfamily.dat | 38 ++ src/include/catalog/pg_proc.dat | 34 + src/include/catalog/pg_type.dat | 7 +- src/test/regress/expected/brin_bloom.out | 428 ++++++++++++ src/test/regress/expected/opr_sanity.out | 3 +- src/test/regress/expected/psql.out | 3 +- src/test/regress/expected/type_sanity.out | 7 +- src/test/regress/parallel_schedule | 5 + src/test/regress/serial_schedule | 1 + src/test/regress/sql/brin_bloom.sql | 376 +++++++++++ 19 files changed, 2555 insertions(+), 7 deletions(-) create mode 100644 src/backend/access/brin/brin_bloom.c create mode 100644 src/test/regress/expected/brin_bloom.out create mode 100644 src/test/regress/sql/brin_bloom.sql diff --git a/doc/src/sgml/brin.sgml b/doc/src/sgml/brin.sgml index 06880c0f7b..58126a5a3c 100644 --- a/doc/src/sgml/brin.sgml +++ b/doc/src/sgml/brin.sgml @@ -115,7 +115,8 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was operator classes store the minimum and the maximum values appearing in the indexed column within the range. The <firstterm>inclusion</firstterm> operator classes store a value which includes the values in the indexed - column within the range. + column within the range. The <firstterm>bloom</firstterm> operator + classes build a Bloom filter for all values in the range. </para> <table id="brin-builtin-opclasses-table"> @@ -128,6 +129,10 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was </row> </thead> <tbody> + <row> + <entry valign="middle"><literal>int8_bloom_ops</literal></entry> + <entry><literal>= (bit,bit)</literal></entry> + </row> <row> <entry valign="middle" morerows="4"><literal>bit_minmax_ops</literal></entry> <entry><literal>= (bit,bit)</literal></entry> @@ -154,6 +159,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>|&> (box,box)</literal></entry></row> <row><entry><literal>|>> (box,box)</literal></entry></row> + <row> + <entry valign="middle"><literal>bpchar_bloom_ops</literal></entry> + <entry><literal>= (character,character)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>bpchar_minmax_ops</literal></entry> <entry><literal>= (character,character)</literal></entry> @@ -163,6 +173,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (character,character)</literal></entry></row> <row><entry><literal>>= (character,character)</literal></entry></row> + <row> + <entry valign="middle"><literal>bytea_bloom_ops</literal></entry> + <entry><literal>= (bytea,bytea)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>bytea_minmax_ops</literal></entry> <entry><literal>= (bytea,bytea)</literal></entry> @@ -172,6 +187,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (bytea,bytea)</literal></entry></row> <row><entry><literal>>= (bytea,bytea)</literal></entry></row> + <row> + <entry valign="middle"><literal>char_bloom_ops</literal></entry> + <entry><literal>= ("char","char")</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>char_minmax_ops</literal></entry> <entry><literal>= ("char","char")</literal></entry> @@ -181,6 +201,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> ("char","char")</literal></entry></row> <row><entry><literal>>= ("char","char")</literal></entry></row> + <row> + <entry valign="middle"><literal>date_bloom_ops</literal></entry> + <entry><literal>= (date,date)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>date_minmax_ops</literal></entry> <entry><literal>= (date,date)</literal></entry> @@ -190,6 +215,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (date,date)</literal></entry></row> <row><entry><literal>>= (date,date)</literal></entry></row> + <row> + <entry valign="middle"><literal>float4_bloom_ops</literal></entry> + <entry><literal>= (float4,float4)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>float4_minmax_ops</literal></entry> <entry><literal>= (float4,float4)</literal></entry> @@ -199,6 +229,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal><= (float4,float4)</literal></entry></row> <row><entry><literal>>= (float4,float4)</literal></entry></row> + <row> + <entry valign="middle"><literal>float8_bloom_ops</literal></entry> + <entry><literal>= (float8,float8)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>float8_minmax_ops</literal></entry> <entry><literal>= (float8,float8)</literal></entry> @@ -218,6 +253,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>= (inet,inet)</literal></entry></row> <row><entry><literal>&& (inet,inet)</literal></entry></row> + <row> + <entry valign="middle"><literal>inet_bloom_ops</literal></entry> + <entry><literal>= (inet,inet)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>inet_minmax_ops</literal></entry> <entry><literal>= (inet,inet)</literal></entry> @@ -227,6 +267,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (inet,inet)</literal></entry></row> <row><entry><literal>>= (inet,inet)</literal></entry></row> + <row> + <entry valign="middle"><literal>int2_bloom_ops</literal></entry> + <entry><literal>= (int2,int2)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>int2_minmax_ops</literal></entry> <entry><literal>= (int2,int2)</literal></entry> @@ -236,6 +281,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal><= (int2,int2)</literal></entry></row> <row><entry><literal>>= (int2,int2)</literal></entry></row> + <row> + <entry valign="middle"><literal>int4_bloom_ops</literal></entry> + <entry><literal>= (int4,int4)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>int4_minmax_ops</literal></entry> <entry><literal>= (int4,int4)</literal></entry> @@ -245,6 +295,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal><= (int4,int4)</literal></entry></row> <row><entry><literal>>= (int4,int4)</literal></entry></row> + <row> + <entry valign="middle"><literal>int8_bloom_ops</literal></entry> + <entry><literal>= (bigint,bigint)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>int8_minmax_ops</literal></entry> <entry><literal>= (bigint,bigint)</literal></entry> @@ -254,6 +309,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal><= (bigint,bigint)</literal></entry></row> <row><entry><literal>>= (bigint,bigint)</literal></entry></row> + <row> + <entry valign="middle"><literal>interval_bloom_ops</literal></entry> + <entry><literal>= (interval,interval)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>interval_minmax_ops</literal></entry> <entry><literal>= (interval,interval)</literal></entry> @@ -263,6 +323,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (interval,interval)</literal></entry></row> <row><entry><literal>>= (interval,interval)</literal></entry></row> + <row> + <entry valign="middle"><literal>macaddr_bloom_ops</literal></entry> + <entry><literal>= (macaddr,macaddr)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>macaddr_minmax_ops</literal></entry> <entry><literal>= (macaddr,macaddr)</literal></entry> @@ -272,6 +337,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (macaddr,macaddr)</literal></entry></row> <row><entry><literal>>= (macaddr,macaddr)</literal></entry></row> + <row> + <entry valign="middle"><literal>macaddr8_bloom_ops</literal></entry> + <entry><literal>= (macaddr8,macaddr8)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>macaddr8_minmax_ops</literal></entry> <entry><literal>= (macaddr8,macaddr8)</literal></entry> @@ -281,6 +351,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (macaddr8,macaddr8)</literal></entry></row> <row><entry><literal>>= (macaddr8,macaddr8)</literal></entry></row> + <row> + <entry valign="middle"><literal>name_bloom_ops</literal></entry> + <entry><literal>= (name,name)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>name_minmax_ops</literal></entry> <entry><literal>= (name,name)</literal></entry> @@ -290,6 +365,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (name,name)</literal></entry></row> <row><entry><literal>>= (name,name)</literal></entry></row> + <row> + <entry valign="middle"><literal>numeric_bloom_ops</literal></entry> + <entry><literal>= (numeric,numeric)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>numeric_minmax_ops</literal></entry> <entry><literal>= (numeric,numeric)</literal></entry> @@ -299,6 +379,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (numeric,numeric)</literal></entry></row> <row><entry><literal>>= (numeric,numeric)</literal></entry></row> + <row> + <entry valign="middle"><literal>oid_bloom_ops</literal></entry> + <entry><literal>= (oid,oid)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>oid_minmax_ops</literal></entry> <entry><literal>= (oid,oid)</literal></entry> @@ -308,6 +393,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal><= (oid,oid)</literal></entry></row> <row><entry><literal>>= (oid,oid)</literal></entry></row> + <row> + <entry valign="middle"><literal>pg_lsn_bloom_ops</literal></entry> + <entry><literal>= (pg_lsn,pg_lsn)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>pg_lsn_minmax_ops</literal></entry> <entry><literal>= (pg_lsn,pg_lsn)</literal></entry> @@ -335,6 +425,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>&> (anyrange,anyrange)</literal></entry></row> <row><entry><literal>-|- (anyrange,anyrange)</literal></entry></row> + <row> + <entry valign="middle"><literal>text_bloom_ops</literal></entry> + <entry><literal>= (text,text)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>text_minmax_ops</literal></entry> <entry><literal>= (text,text)</literal></entry> @@ -344,6 +439,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (text,text)</literal></entry></row> <row><entry><literal>>= (text,text)</literal></entry></row> + <row> + <entry valign="middle"><literal>tid_bloom_ops</literal></entry> + <entry><literal>= (tid,tid)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>tid_minmax_ops</literal></entry> <entry><literal>= (tid,tid)</literal></entry> @@ -353,6 +453,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal><= (tid,tid)</literal></entry></row> <row><entry><literal>>= (tid,tid)</literal></entry></row> + <row> + <entry valign="middle"><literal>timestamp_bloom_ops</literal></entry> + <entry><literal>= (timestamp,timestamp)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>timestamp_minmax_ops</literal></entry> <entry><literal>= (timestamp,timestamp)</literal></entry> @@ -362,6 +467,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (timestamp,timestamp)</literal></entry></row> <row><entry><literal>>= (timestamp,timestamp)</literal></entry></row> + <row> + <entry valign="middle"><literal>timestamptz_bloom_ops</literal></entry> + <entry><literal>= (timestamptz,timestamptz)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>timestamptz_minmax_ops</literal></entry> <entry><literal>= (timestamptz,timestamptz)</literal></entry> @@ -371,6 +481,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (timestamptz,timestamptz)</literal></entry></row> <row><entry><literal>>= (timestamptz,timestamptz)</literal></entry></row> + <row> + <entry valign="middle"><literal>time_bloom_ops</literal></entry> + <entry><literal>= (time,time)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>time_minmax_ops</literal></entry> <entry><literal>= (time,time)</literal></entry> @@ -380,6 +495,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (time,time)</literal></entry></row> <row><entry><literal>>= (time,time)</literal></entry></row> + <row> + <entry valign="middle"><literal>timetz_bloom_ops</literal></entry> + <entry><literal>= (timetz,timetz)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>timetz_minmax_ops</literal></entry> <entry><literal>= (timetz,timetz)</literal></entry> @@ -389,6 +509,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <row><entry><literal>> (timetz,timetz)</literal></entry></row> <row><entry><literal>>= (timetz,timetz)</literal></entry></row> + <row> + <entry valign="middle"><literal>uuid_bloom_ops</literal></entry> + <entry><literal>= (uuid,uuid)</literal></entry> + </row> + <row> <entry valign="middle" morerows="4"><literal>uuid_minmax_ops</literal></entry> <entry><literal>= (uuid,uuid)</literal></entry> @@ -409,6 +534,55 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was </tbody> </tgroup> </table> + + <sect2 id="brin-builtin-opclasses--parameters"> + <title>Operator Class Parameters</title> + + <para> + Some of the built-in operator classes allow specifying parameters affecting + behavior of the operator class. Each operator class has its own set of + allowed parameters. Only the <literal>bloom</literal> operator class + allows specifying parameters: + </para> + + <para> + <acronym>bloom</acronym> operator classes accept these parameters: + </para> + + <variablelist> + <varlistentry> + <term><literal>n_distinct_per_range</literal></term> + <listitem> + <para> + Defines the estimated number of distinct non-null values in the block + range, used by <acronym>BRIN</acronym> bloom indexes for sizing of the + Bloom filter. It behaves similarly to <literal>n_distinct</literal> option + for <xref linkend="sql-altertable"/>. When set to a positive value, + each block range is assumed to contain this number of distinct non-null + values. When set to a negative value, which must be greater than or + equal to -1, the number of distinct non-null is assumed linear with + the maximum possible number of tuples in the block range (about 290 + rows per block). The default value is <literal>-0.1</literal>, and + the minimum number of distinct non-null values is <literal>16</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>false_positive_rate</literal></term> + <listitem> + <para> + Defines the desired false positive rate used by <acronym>BRIN</acronym> + bloom indexes for sizing of the Bloom filter. The values must be + between 0.0001 and 0.25. The default value is 0.01, which is 1% false + positive rate. + </para> + </listitem> + </varlistentry> + + </variablelist> + </sect2> + </sect1> <sect1 id="brin-extensibility"> @@ -779,6 +953,60 @@ typedef struct BrinOpcInfo function can improve index performance. </para> + <para> + To write an operator class for a data type that implements only an equality + operator and supports hashing, it is possible to use the bloom support procedures + alongside the corresponding operators, as shown in + <xref linkend="brin-extensibility-bloom-table"/>. + All operator class members (procedures and operators) are mandatory. + </para> + + <table id="brin-extensibility-bloom-table"> + <title>Procedure and Support Numbers for Bloom Operator Classes</title> + <tgroup cols="2"> + <thead> + <row> + <entry>Operator class member</entry> + <entry>Object</entry> + </row> + </thead> + <tbody> + <row> + <entry>Support Procedure 1</entry> + <entry>internal function <function>brin_bloom_opcinfo()</function></entry> + </row> + <row> + <entry>Support Procedure 2</entry> + <entry>internal function <function>brin_bloom_add_value()</function></entry> + </row> + <row> + <entry>Support Procedure 3</entry> + <entry>internal function <function>brin_bloom_consistent()</function></entry> + </row> + <row> + <entry>Support Procedure 4</entry> + <entry>internal function <function>brin_bloom_union()</function></entry> + </row> + <row> + <entry>Support Procedure 11</entry> + <entry>function to compute hash of an element</entry> + </row> + <row> + <entry>Operator Strategy 1</entry> + <entry>operator equal-to</entry> + </row> + </tbody> + </tgroup> + </table> + + <para> + Support procedure numbers 1-10 are reserved for the BRIN internal + functions, so the SQL level functions start with number 11. Support + function number 11 is the main function required to build the index. + It should accept one argument with the same data type as the operator class, + and return a hash of the value. + </para> + <para> Both minmax and inclusion operator classes support cross-data-type operators, though with these the dependencies become more complicated. diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/create_index.sgml index 51b4d57939..4a29935785 100644 --- a/doc/src/sgml/ref/create_index.sgml +++ b/doc/src/sgml/ref/create_index.sgml @@ -581,6 +581,7 @@ CREATE [ UNIQUE ] INDEX [ CONCURRENTLY ] [ [ IF NOT EXISTS ] <replaceable class= </para> </listitem> </varlistentry> + </variablelist> </refsect2> diff --git a/src/backend/access/brin/Makefile b/src/backend/access/brin/Makefile index 468e1e289a..6b56131215 100644 --- a/src/backend/access/brin/Makefile +++ b/src/backend/access/brin/Makefile @@ -14,6 +14,7 @@ include $(top_builddir)/src/Makefile.global OBJS = \ brin.o \ + brin_bloom.o \ brin_inclusion.o \ brin_minmax.o \ brin_pageops.o \ diff --git a/src/backend/access/brin/brin_bloom.c b/src/backend/access/brin/brin_bloom.c new file mode 100644 index 0000000000..4494484b3b --- /dev/null +++ b/src/backend/access/brin/brin_bloom.c @@ -0,0 +1,787 @@ +/* + * brin_bloom.c + * Implementation of Bloom opclass for BRIN + * + * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * A BRIN opclass summarizing page range into a bloom filter. + * + * Bloom filters allow efficient testing whether a given page range contains + * a particular value. Therefore, if we summarize each page range into a + * bloom filter, we can easily and cheaply test whether it contains values + * we get later. + * + * The index only supports equality operators, similarly to hash indexes. + * BRIN bloom indexes are however much smaller, and support only bitmap + * scans. + * + * Note: Don't confuse this with bloom indexes, implemented in a contrib + * module. That extension implements an entirely new AM, building a bloom + * filter on multiple columns in a single row. This opclass works with an + * existing AM (BRIN) and builds bloom filter on a column. + * + * + * values vs. hashes + * ----------------- + * + * The original column values are not used directly, but are first hashed + * using the regular type-specific hash function, producing a uint32 hash. + * And this hash value is then added to the summary - i.e. it's hashed + * again and added to the bloom filter. + * + * This allows the code to treat all data types (byval/byref/...) the same + * way, with only minimal space requirements, because we're working with + * hashes and not the original values. Everything is uint32. + * + * Of course, this assumes the built-in hash function is reasonably good, + * without too many collisions etc. But that does seem to be the case, at + * least based on past experience. After all, the same hash functions are + * used for hash indexes, hash partitioning and so on. + * + * + * sizing the bloom filter + * ----------------------- + * + * Size of a bloom filter depends on the number of distinct values we will + * store in it, and the desired false positive rate. The higher the number + * of distinct values and/or the lower the false positive rate, the larger + * the bloom filter. On the other hand, we want to keep the index as small + * as possible - that's one of the basic advantages of BRIN indexes. + * + * Although the number of distinct elements (in a page range) depends on + * the data, we can consider it fixed. This simplifies the trade-off to + * just false positive rate vs. size. + * + * At the page range level, false positive rate is a probability the bloom + * filter matches a random value. For the whole index (with sufficiently + * many page ranges) it represents the fraction of the index ranges (and + * thus fraction of the table to be scanned) matching the random value. + * + * Furthermore, the size of the bloom filter is subject to implementation + * limits - it has to fit onto a single index page (8kB by default). As + * the bitmap is inherently random, compression can't reliably help here. + * To reduce the size of a filter (to fit to a page), we have to either + * accept higher false positive rate (undesirable), or reduce the number + * of distinct items to be stored in the filter. We can't alter the input + * data, of course, but we may make the BRIN page ranges smaller - instead + * of the default 128 pages (1MB) we may build index with 16-page ranges, + * or something like that. This does help even for random data sets, as + * the number of rows per heap page is limited (to ~290 with very narrow + * tables, likely ~20 in practice). + * + * Of course, good sizing decisions depend on having the necessary data, + * i.e. number of distinct values in a page range (of a given size) and + * table size (to estimate cost change due to change in false positive + * rate due to having larger index vs. scanning larger indexes). We may + * not have that data - for example when building an index on empty table + * it's not really possible. And for some data we only have estimates for + * the whole table and we can only estimate per-range values (ndistinct). + * + * Another challenge is that while the bloom filter is per-column, it's + * the whole index tuple that has to fit into a page. And for multi-column + * indexes that may include pieces we have no control over (not necessarily + * bloom filters, the other columns may use other BRIN opclasses). So it's + * not entirely clear how to distrubute the space between those columns. + * + * The current logic, implemented in brin_bloom_get_ndistinct, attempts to + * make some basic sizing decisions, based on the size of BRIN ranges, and + * the maximum number of rows per range. + * + * + * IDENTIFICATION + * src/backend/access/brin/brin_bloom.c + */ +#include "postgres.h" + +#include "access/genam.h" +#include "access/brin.h" +#include "access/brin_internal.h" +#include "access/brin_page.h" +#include "access/brin_tuple.h" +#include "access/hash.h" +#include "access/htup_details.h" +#include "access/reloptions.h" +#include "access/stratnum.h" +#include "catalog/pg_type.h" +#include "catalog/pg_amop.h" +#include "utils/builtins.h" +#include "utils/datum.h" +#include "utils/lsyscache.h" +#include "utils/rel.h" +#include "utils/syscache.h" + +#include <math.h> + +#define BloomEqualStrategyNumber 1 + +/* + * Additional SQL level support functions + * + * Procedure numbers must not use values reserved for BRIN itself; see + * brin_internal.h. + */ +#define BLOOM_MAX_PROCNUMS 1 /* maximum support procs we need */ +#define PROCNUM_HASH 11 /* required */ + +/* + * Subtract this from procnum to obtain index in BloomOpaque arrays + * (Must be equal to minimum of private procnums). + */ +#define PROCNUM_BASE 11 + +/* + * Storage type for BRIN's reloptions. + */ +typedef struct BloomOptions +{ + int32 vl_len_; /* varlena header (do not touch directly!) */ + double nDistinctPerRange; /* number of distinct values per range */ + double falsePositiveRate; /* false positive for bloom filter */ +} BloomOptions; + +/* + * The current min value (16) is somewhat arbitrary, but it's based + * on the fact that the filter header is ~20B alone, which is about + * the same as the filter bitmap for 16 distinct items with 1% false + * positive rate. So by allowing lower values we'd not gain much. In + * any case, the min should not be larger than MaxHeapTuplesPerPage + * (~290), which is the theoretical maximum for single-page ranges. + */ +#define BLOOM_MIN_NDISTINCT_PER_RANGE 16 + +/* + * Used to determine number of distinct items, based on the number of rows + * in a page range. The 10% is somewhat similar to what estimate_num_groups + * does, so we use the same factor here. + */ +#define BLOOM_DEFAULT_NDISTINCT_PER_RANGE -0.1 /* 10% of values */ + +/* + * Allowed range and default value for the false positive range. The exact + * values are somewhat arbitrary, but were chosen considering the various + * parameters (size of filter vs. page size, etc.). + * + * The lower the false-positive rate, the more accurate the filter is, but + * it also gets larger - at some point this eliminates the main advantage + * of BRIN indexes, which is the tiny size. At 0.01% the index is about + * 10% of the table (assuming 290 distinct values per 8kB page). + * + * On the other hand, as the false-positive rate increases, larger part of + * the table has to be scanned due to mismatches - at 25% we're probably + * close to sequential scan being cheaper. + */ +#define BLOOM_MIN_FALSE_POSITIVE_RATE 0.0001 /* 0.01% fp rate */ +#define BLOOM_MAX_FALSE_POSITIVE_RATE 0.25 /* 25% fp rate */ +#define BLOOM_DEFAULT_FALSE_POSITIVE_RATE 0.01 /* 1% fp rate */ + +#define BloomGetNDistinctPerRange(opts) \ + ((opts) && (((BloomOptions *) (opts))->nDistinctPerRange != 0) ? \ + (((BloomOptions *) (opts))->nDistinctPerRange) : \ + BLOOM_DEFAULT_NDISTINCT_PER_RANGE) + +#define BloomGetFalsePositiveRate(opts) \ + ((opts) && (((BloomOptions *) (opts))->falsePositiveRate != 0.0) ? \ + (((BloomOptions *) (opts))->falsePositiveRate) : \ + BLOOM_DEFAULT_FALSE_POSITIVE_RATE) + + +#define BloomMaxFilterSize \ + MAXALIGN_DOWN(BLCKSZ - \ + (MAXALIGN(SizeOfPageHeaderData + \ + sizeof(ItemIdData)) + \ + MAXALIGN(sizeof(BrinSpecialSpace)) + \ + SizeOfBrinTuple)) + + +/* + * Bloom Filter + * + * Represents a bloom filter, built on hashes of the indexed values. That is, + * we compute a uint32 hash of the value, and then store this hash into the + * bloom filter (and compute additional hashes on it). + * + * To calculate the additional hashes (treating the uint32 hash as an input + * value), we use the approach with two hash functions from this paper: + * + * Less Hashing, Same Performance:Building a Better Bloom Filter + * Adam Kirsch, Michael Mitzenmacher†, Harvard School of Engineering and + * Applied Sciences, Cambridge, Massachusetts [DOI 10.1002/rsa.20208] + * + * The two hash functions are calculated using hard-coded seeds. + * + * XXX We could implement "sparse" bloom filters, keeping only the bytes + * that are not entirely 0. But while indexes don't support TOAST, the + * varlena can still be compressed. So this seems unnecessary. + * + * XXX We can also watch the number of bits set in the bloom filter, and + * then stop using it (and not store the bitmap, to save space) when the + * false positive rate gets too high. But even if the false positive rate + * exceeds the desired value, it still can eliminate some page ranges. + */ +typedef struct BloomFilter +{ + /* varlena header (do not touch directly!) */ + int32 vl_len_; + + /* space for various flags (unused for now) */ + uint16 flags; + + /* fields for the HASHED phase */ + uint8 nhashes; /* number of hash functions */ + uint32 nbits; /* number of bits in the bitmap (size) */ + uint32 nbits_set; /* number of bits set to 1 */ + + /* data of the bloom filter */ + char data[FLEXIBLE_ARRAY_MEMBER]; + +} BloomFilter; + + +/* + * bloom_init + * Initialize the Bloom Filter, allocate all the memory. + * + * The filter is initialized with optimal size for ndistinct expected + * values, and requested false positive rate. The filter is stored as + * varlena. + */ +static BloomFilter * +bloom_init(int ndistinct, double false_positive_rate) +{ + Size len; + BloomFilter *filter; + + int nbits; /* size of filter / number of bits */ + int nbytes; /* size of filter / number of bytes */ + + double k; /* number of hash functions */ + + Assert(ndistinct > 0); + Assert((false_positive_rate > 0) && (false_positive_rate < 1.0)); + + /* sizing bloom filter: -(n * ln(p)) / (ln(2))^2 */ + nbits = ceil(-(ndistinct * log(false_positive_rate)) / pow(log(2.0), 2)); + + /* round m to whole bytes */ + nbytes = ((nbits + 7) / 8); + nbits = nbytes * 8; + + /* + * Reject filters that are obviously too large to store on a page. + * + * Initially the bloom filter is just zeroes and so very compressible, but + * as we add values it gets more and more random, and so less and less + * compressible. So initially everything fits on the page, but we might + * get surprising failures later - we want to prevent that, so we reject + * bloom filter that are obviously too large. + * + * XXX It's not uncommon to oversize the bloom filter a bit, to defend + * against unexpected data anomalies (parts of table with more distinct + * values per range etc.). But we still need to make sure even the + * oversized filter fits on page, if such need arises. + * + * XXX This check is not perfect, because the index may have multiple + * filters that are small individually, but too large when combined. + */ + if (nbytes > BloomMaxFilterSize) + elog(ERROR, "the bloom filter is too large (%d > %zu)", nbytes, + BloomMaxFilterSize); + + /* + * round(log(2.0) * m / ndistinct), but assume round() may not be + * available on Windows + */ + k = log(2.0) * nbits / ndistinct; + k = (k - floor(k) >= 0.5) ? ceil(k) : floor(k); + + /* + * We allocate the whole filter. Most of it is going to be 0 bits, so the + * varlena is easy to compress. + */ + len = offsetof(BloomFilter, data) + nbytes; + + filter = (BloomFilter *) palloc0(len); + + filter->flags = 0; + filter->nhashes = (int) k; + filter->nbits = nbits; + + SET_VARSIZE(filter, len); + + return filter; +} + + +/* + * bloom_add_value + * Add value to the bloom filter. + */ +static BloomFilter * +bloom_add_value(BloomFilter * filter, uint32 value, bool *updated) +{ + int i; + uint64 h1, + h2; + + /* compute the hashes, used for the bloom filter */ + h1 = DatumGetUInt64(hash_uint32_extended(value, 0x71d924af)) % filter->nbits; + h2 = DatumGetUInt64(hash_uint32_extended(value, 0xba48b314)) % filter->nbits; + + /* compute the requested number of hashes */ + for (i = 0; i < filter->nhashes; i++) + { + /* h1 + h2 + f(i) */ + uint32 h = (h1 + i * h2) % filter->nbits; + uint32 byte = (h / 8); + uint32 bit = (h % 8); + + /* if the bit is not set, set it and remember we did that */ + if (!(filter->data[byte] & (0x01 << bit))) + { + filter->data[byte] |= (0x01 << bit); + filter->nbits_set++; + if (updated) + *updated = true; + } + } + + return filter; +} + + +/* + * bloom_contains_value + * Check if the bloom filter contains a particular value. + */ +static bool +bloom_contains_value(BloomFilter * filter, uint32 value) +{ + int i; + uint64 h1, + h2; + + /* calculate the two hashes */ + h1 = DatumGetUInt64(hash_uint32_extended(value, 0x71d924af)) % filter->nbits; + h2 = DatumGetUInt64(hash_uint32_extended(value, 0xba48b314)) % filter->nbits; + + /* compute the requested number of hashes */ + for (i = 0; i < filter->nhashes; i++) + { + /* h1 + h2 + f(i) */ + uint32 h = (h1 + i * h2) % filter->nbits; + uint32 byte = (h / 8); + uint32 bit = (h % 8); + + /* if the bit is not set, the value is not there */ + if (!(filter->data[byte] & (0x01 << bit))) + return false; + } + + /* all hashes found in bloom filter */ + return true; +} + +typedef struct BloomOpaque +{ + /* + * XXX At this point we only need a single proc (to compute the hash), but + * let's keep the array just like inclusion and minman opclasses, for + * consistency. We may need additional procs in the future. + */ + FmgrInfo extra_procinfos[BLOOM_MAX_PROCNUMS]; + bool extra_proc_missing[BLOOM_MAX_PROCNUMS]; +} BloomOpaque; + +static FmgrInfo *bloom_get_procinfo(BrinDesc *bdesc, uint16 attno, + uint16 procnum); + + +Datum +brin_bloom_opcinfo(PG_FUNCTION_ARGS) +{ + BrinOpcInfo *result; + + /* + * opaque->strategy_procinfos is initialized lazily; here it is set to + * all-uninitialized by palloc0 which sets fn_oid to InvalidOid. + * + * bloom indexes only store the filter as a single BYTEA column + */ + + result = palloc0(MAXALIGN(SizeofBrinOpcInfo(1)) + + sizeof(BloomOpaque)); + result->oi_nstored = 1; + result->oi_regular_nulls = true; + result->oi_opaque = (BloomOpaque *) + MAXALIGN((char *) result + SizeofBrinOpcInfo(1)); + result->oi_typcache[0] = lookup_type_cache(PG_BRIN_BLOOM_SUMMARYOID, 0); + + PG_RETURN_POINTER(result); +} + +/* + * brin_bloom_get_ndistinct + * Determine the ndistinct value used to size bloom filter. + * + * Adjust the ndistinct value based on the pagesPerRange value. First, + * if it's negative, it's assumed to be relative to maximum number of + * tuples in the range (assuming each page gets MaxHeapTuplesPerPage + * tuples, which is likely a significant over-estimate). We also clamp + * the value, not to over-size the bloom filter unnecessarily. + * + * XXX We can only do this when the pagesPerRange value was supplied. + * If it wasn't, it has to be a read-only access to the index, in which + * case we don't really care. But perhaps we should fall-back to the + * default pagesPerRange value? + * + * XXX We might also fetch info about ndistinct estimate for the column, + * and compute the expected number of distinct values in a range. But + * that may be tricky due to data being sorted in various ways, so it + * seems better to rely on the upper estimate. + * + * XXX We might also calculate a better estimate of rows per BRIN range, + * instead of using MaxHeapTuplesPerPage (which probably produces values + * much higher than reality). + */ +static int +brin_bloom_get_ndistinct(BrinDesc *bdesc, BloomOptions *opts) +{ + double ndistinct; + double maxtuples; + BlockNumber pagesPerRange; + + pagesPerRange = BrinGetPagesPerRange(bdesc->bd_index); + ndistinct = BloomGetNDistinctPerRange(opts); + + Assert(BlockNumberIsValid(pagesPerRange)); + + maxtuples = MaxHeapTuplesPerPage * pagesPerRange; + + /* + * Similarly to n_distinct, negative values are relative - in this case to + * maximum number of tuples in the page range (maxtuples). + */ + if (ndistinct < 0) + ndistinct = (-ndistinct) * maxtuples; + + /* + * Positive values are to be used directly, but we still apply a couple of + * safeties no to use unreasonably small bloom filters. + */ + ndistinct = Max(ndistinct, BLOOM_MIN_NDISTINCT_PER_RANGE); + + /* + * And don't use more than the maximum possible number of tuples, in the + * range, which would be entirely wasteful. + */ + ndistinct = Min(ndistinct, maxtuples); + + return (int) ndistinct; +} + +/* + * Examine the given index tuple (which contains partial status of a certain + * page range) by comparing it to the given value that comes from another heap + * tuple. If the new value is outside the bloom filter specified by the + * existing tuple values, update the index tuple and return true. Otherwise, + * return false and do not modify in this case. + */ +Datum +brin_bloom_add_value(PG_FUNCTION_ARGS) +{ + BrinDesc *bdesc = (BrinDesc *) PG_GETARG_POINTER(0); + BrinValues *column = (BrinValues *) PG_GETARG_POINTER(1); + Datum newval = PG_GETARG_DATUM(2); + bool isnull PG_USED_FOR_ASSERTS_ONLY = PG_GETARG_DATUM(3); + BloomOptions *opts = (BloomOptions *) PG_GET_OPCLASS_OPTIONS(); + Oid colloid = PG_GET_COLLATION(); + FmgrInfo *hashFn; + uint32 hashValue; + bool updated = false; + AttrNumber attno; + BloomFilter *filter; + + Assert(!isnull); + + attno = column->bv_attno; + + /* + * If this is the first non-null value, we need to initialize the bloom + * filter. Otherwise just extract the existing bloom filter from + * BrinValues. + */ + if (column->bv_allnulls) + { + filter = bloom_init(brin_bloom_get_ndistinct(bdesc, opts), + BloomGetFalsePositiveRate(opts)); + column->bv_values[0] = PointerGetDatum(filter); + column->bv_allnulls = false; + updated = true; + } + else + filter = (BloomFilter *) PG_DETOAST_DATUM(column->bv_values[0]); + + /* + * Compute the hash of the new value, using the supplied hash function, + * and then add the hash value to the bloom filter. + */ + hashFn = bloom_get_procinfo(bdesc, attno, PROCNUM_HASH); + + hashValue = DatumGetUInt32(FunctionCall1Coll(hashFn, colloid, newval)); + + filter = bloom_add_value(filter, hashValue, &updated); + + column->bv_values[0] = PointerGetDatum(filter); + + PG_RETURN_BOOL(updated); +} + +/* + * Given an index tuple corresponding to a certain page range and a scan key, + * return whether the scan key is consistent with the index tuple's bloom + * filter. Return true if so, false otherwise. + */ +Datum +brin_bloom_consistent(PG_FUNCTION_ARGS) +{ + BrinDesc *bdesc = (BrinDesc *) PG_GETARG_POINTER(0); + BrinValues *column = (BrinValues *) PG_GETARG_POINTER(1); + ScanKey *keys = (ScanKey *) PG_GETARG_POINTER(2); + int nkeys = PG_GETARG_INT32(3); + Oid colloid = PG_GET_COLLATION(); + AttrNumber attno; + Datum value; + Datum matches; + FmgrInfo *finfo; + uint32 hashValue; + BloomFilter *filter; + int keyno; + + filter = (BloomFilter *) PG_DETOAST_DATUM(column->bv_values[0]); + + Assert(filter); + + matches = true; + + for (keyno = 0; keyno < nkeys; keyno++) + { + ScanKey key = keys[keyno]; + + /* NULL keys are handled and filtered-out in bringetbitmap */ + Assert(!(key->sk_flags & SK_ISNULL)); + + attno = key->sk_attno; + value = key->sk_argument; + + switch (key->sk_strategy) + { + case BloomEqualStrategyNumber: + + /* + * In the equality case (WHERE col = someval), we want to + * return the current page range if the minimum value in the + * range <= scan key, and the maximum value >= scan key. + */ + finfo = bloom_get_procinfo(bdesc, attno, PROCNUM_HASH); + + hashValue = DatumGetUInt32(FunctionCall1Coll(finfo, colloid, value)); + matches &= bloom_contains_value(filter, hashValue); + + break; + default: + /* shouldn't happen */ + elog(ERROR, "invalid strategy number %d", key->sk_strategy); + matches = 0; + break; + } + + if (!matches) + break; + } + + PG_RETURN_DATUM(matches); +} + +/* + * Given two BrinValues, update the first of them as a union of the summary + * values contained in both. The second one is untouched. + * + * XXX We assume the bloom filters have the same parameters for now. In the + * future we should have 'can union' function, to decide if we can combine + * two particular bloom filters. + */ +Datum +brin_bloom_union(PG_FUNCTION_ARGS) +{ + int i; + int nbytes; + BrinValues *col_a = (BrinValues *) PG_GETARG_POINTER(1); + BrinValues *col_b = (BrinValues *) PG_GETARG_POINTER(2); + BloomFilter *filter_a; + BloomFilter *filter_b; + + Assert(col_a->bv_attno == col_b->bv_attno); + Assert(!col_a->bv_allnulls && !col_b->bv_allnulls); + + filter_a = (BloomFilter *) PG_DETOAST_DATUM(col_a->bv_values[0]); + filter_b = (BloomFilter *) PG_DETOAST_DATUM(col_b->bv_values[0]); + + /* make sure the filters use the same parameters */ + Assert(filter_a && filter_b); + Assert(filter_a->nbits == filter_b->nbits); + Assert(filter_a->nhashes == filter_b->nhashes); + Assert((filter_a->nbits > 0) && (filter_a->nbits % 8 == 0)); + + nbytes = (filter_a->nbits) / 8; + + /* simply OR the bitmaps */ + for (i = 0; i < nbytes; i++) + filter_a->data[i] |= filter_b->data[i]; + + PG_RETURN_VOID(); +} + +/* + * Cache and return inclusion opclass support procedure + * + * Return the procedure corresponding to the given function support number + * or null if it does not exist. + */ +static FmgrInfo * +bloom_get_procinfo(BrinDesc *bdesc, uint16 attno, uint16 procnum) +{ + BloomOpaque *opaque; + uint16 basenum = procnum - PROCNUM_BASE; + + /* + * We cache these in the opaque struct, to avoid repetitive syscache + * lookups. + */ + opaque = (BloomOpaque *) bdesc->bd_info[attno - 1]->oi_opaque; + + /* + * If we already searched for this proc and didn't find it, don't bother + * searching again. + */ + if (opaque->extra_proc_missing[basenum]) + return NULL; + + if (opaque->extra_procinfos[basenum].fn_oid == InvalidOid) + { + if (RegProcedureIsValid(index_getprocid(bdesc->bd_index, attno, + procnum))) + { + fmgr_info_copy(&opaque->extra_procinfos[basenum], + index_getprocinfo(bdesc->bd_index, attno, procnum), + bdesc->bd_context); + } + else + { + opaque->extra_proc_missing[basenum] = true; + return NULL; + } + } + + return &opaque->extra_procinfos[basenum]; +} + +Datum +brin_bloom_options(PG_FUNCTION_ARGS) +{ + local_relopts *relopts = (local_relopts *) PG_GETARG_POINTER(0); + + init_local_reloptions(relopts, sizeof(BloomOptions)); + + add_local_real_reloption(relopts, "n_distinct_per_range", + "number of distinct items expected in a BRIN page range", + BLOOM_DEFAULT_NDISTINCT_PER_RANGE, + -1.0, INT_MAX, offsetof(BloomOptions, nDistinctPerRange)); + + add_local_real_reloption(relopts, "false_positive_rate", + "desired false-positive rate for the bloom filters", + BLOOM_DEFAULT_FALSE_POSITIVE_RATE, + BLOOM_MIN_FALSE_POSITIVE_RATE, + BLOOM_MAX_FALSE_POSITIVE_RATE, + offsetof(BloomOptions, falsePositiveRate)); + + PG_RETURN_VOID(); +} + +/* + * brin_bloom_summary_in + * - input routine for type brin_bloom_summary. + * + * brin_bloom_summary is only used internally to represent summaries + * in BRIN bloom indexes, so it has no operations of its own, and we + * disallow input too. + */ +Datum +brin_bloom_summary_in(PG_FUNCTION_ARGS) +{ + /* + * brin_bloom_summary stores the data in binary form and parsing text + * input is not needed, so disallow this. + */ + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot accept a value of type %s", "pg_brin_bloom_summary"))); + + PG_RETURN_VOID(); /* keep compiler quiet */ +} + + +/* + * brin_bloom_summary_out + * - output routine for type brin_bloom_summary. + * + * BRIN bloom summaries are serialized into a bytea value, but we want + * to output something nicer humans can understand. + */ +Datum +brin_bloom_summary_out(PG_FUNCTION_ARGS) +{ + BloomFilter *filter; + StringInfoData str; + + /* detoast the data to get value with a full 4B header */ + filter = (BloomFilter *) PG_DETOAST_DATUM(PG_GETARG_BYTEA_PP(0)); + + initStringInfo(&str); + appendStringInfoChar(&str, '{'); + + appendStringInfo(&str, "mode: hashed nhashes: %u nbits: %u nbits_set: %u", + filter->nhashes, filter->nbits, filter->nbits_set); + + appendStringInfoChar(&str, '}'); + + PG_RETURN_CSTRING(str.data); +} + +/* + * brin_bloom_summary_recv + * - binary input routine for type brin_bloom_summary. + */ +Datum +brin_bloom_summary_recv(PG_FUNCTION_ARGS) +{ + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot accept a value of type %s", "pg_brin_bloom_summary"))); + + PG_RETURN_VOID(); /* keep compiler quiet */ +} + +/* + * brin_bloom_summary_send + * - binary output routine for type brin_bloom_summary. + * + * BRIN bloom summaries are serialized in a bytea value (although the + * type is named differently), so let's just send that. + */ +Datum +brin_bloom_summary_send(PG_FUNCTION_ARGS) +{ + return byteasend(fcinfo); +} diff --git a/src/include/access/brin.h b/src/include/access/brin.h index 4e2be13cd6..0e52d75457 100644 --- a/src/include/access/brin.h +++ b/src/include/access/brin.h @@ -22,6 +22,8 @@ typedef struct BrinOptions int32 vl_len_; /* varlena header (do not touch directly!) */ BlockNumber pagesPerRange; bool autosummarize; + double nDistinctPerRange; /* number of distinct values per range */ + double falsePositiveRate; /* false positive for bloom filter */ } BrinOptions; diff --git a/src/include/access/brin_internal.h b/src/include/access/brin_internal.h index 79440ebe7b..8cc4e532e6 100644 --- a/src/include/access/brin_internal.h +++ b/src/include/access/brin_internal.h @@ -58,6 +58,10 @@ typedef struct BrinDesc /* total number of Datum entries that are stored on-disk for all columns */ int bd_totalstored; + /* parameters for sizing bloom filter (BRIN bloom opclasses) */ + double bd_nDistinctPerRange; + double bd_falsePositiveRange; + /* per-column info; bd_tupdesc->natts entries long */ BrinOpcInfo *bd_info[FLEXIBLE_ARRAY_MEMBER]; } BrinDesc; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..04d678f96a 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1814,6 +1814,11 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# bloom bytea +{ amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, + # minmax "char" { amopfamily => 'brin/char_minmax_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', @@ -1831,6 +1836,11 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# bloom "char" +{ amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', + amopmethod => 'brin' }, + # minmax name { amopfamily => 'brin/name_minmax_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', @@ -1848,6 +1858,11 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# bloom name +{ amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', + amopmethod => 'brin' }, + # minmax integer { amopfamily => 'brin/integer_minmax_ops', amoplefttype => 'int8', @@ -1994,6 +2009,20 @@ amoprighttype => 'int8', amopstrategy => '5', amopopr => '>(int4,int8)', amopmethod => 'brin' }, +# bloom integer + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int8', + amoprighttype => 'int8', amopstrategy => '1', amopopr => '=(int8,int8)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int2', + amoprighttype => 'int2', amopstrategy => '1', amopopr => '=(int2,int2)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int4', + amoprighttype => 'int4', amopstrategy => '1', amopopr => '=(int4,int4)', + amopmethod => 'brin' }, + # minmax text { amopfamily => 'brin/text_minmax_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', @@ -2011,6 +2040,11 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# bloom text +{ amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', + amopmethod => 'brin' }, + # minmax oid { amopfamily => 'brin/oid_minmax_ops', amoplefttype => 'oid', amoprighttype => 'oid', amopstrategy => '1', amopopr => '<(oid,oid)', @@ -2028,6 +2062,11 @@ amoprighttype => 'oid', amopstrategy => '5', amopopr => '>(oid,oid)', amopmethod => 'brin' }, +# bloom oid +{ amopfamily => 'brin/oid_bloom_ops', amoplefttype => 'oid', + amoprighttype => 'oid', amopstrategy => '1', amopopr => '=(oid,oid)', + amopmethod => 'brin' }, + # minmax tid { amopfamily => 'brin/tid_minmax_ops', amoplefttype => 'tid', amoprighttype => 'tid', amopstrategy => '1', amopopr => '<(tid,tid)', @@ -2045,6 +2084,11 @@ amoprighttype => 'tid', amopstrategy => '5', amopopr => '>(tid,tid)', amopmethod => 'brin' }, +# tid oid +{ amopfamily => 'brin/tid_bloom_ops', amoplefttype => 'tid', + amoprighttype => 'tid', amopstrategy => '1', amopopr => '=(tid,tid)', + amopmethod => 'brin' }, + # minmax float (float4, float8) { amopfamily => 'brin/float_minmax_ops', amoplefttype => 'float4', @@ -2111,6 +2155,14 @@ amoprighttype => 'float8', amopstrategy => '5', amopopr => '>(float8,float8)', amopmethod => 'brin' }, +# bloom float +{ amopfamily => 'brin/float_bloom_ops', amoplefttype => 'float4', + amoprighttype => 'float4', amopstrategy => '1', amopopr => '=(float4,float4)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/float_bloom_ops', amoplefttype => 'float8', + amoprighttype => 'float8', amopstrategy => '1', amopopr => '=(float8,float8)', + amopmethod => 'brin' }, + # minmax macaddr { amopfamily => 'brin/macaddr_minmax_ops', amoplefttype => 'macaddr', amoprighttype => 'macaddr', amopstrategy => '1', @@ -2128,6 +2180,11 @@ amoprighttype => 'macaddr', amopstrategy => '5', amopopr => '>(macaddr,macaddr)', amopmethod => 'brin' }, +# bloom macaddr +{ amopfamily => 'brin/macaddr_bloom_ops', amoplefttype => 'macaddr', + amoprighttype => 'macaddr', amopstrategy => '1', + amopopr => '=(macaddr,macaddr)', amopmethod => 'brin' }, + # minmax macaddr8 { amopfamily => 'brin/macaddr8_minmax_ops', amoplefttype => 'macaddr8', amoprighttype => 'macaddr8', amopstrategy => '1', @@ -2145,6 +2202,11 @@ amoprighttype => 'macaddr8', amopstrategy => '5', amopopr => '>(macaddr8,macaddr8)', amopmethod => 'brin' }, +# bloom macaddr8 +{ amopfamily => 'brin/macaddr8_bloom_ops', amoplefttype => 'macaddr8', + amoprighttype => 'macaddr8', amopstrategy => '1', + amopopr => '=(macaddr8,macaddr8)', amopmethod => 'brin' }, + # minmax inet { amopfamily => 'brin/network_minmax_ops', amoplefttype => 'inet', amoprighttype => 'inet', amopstrategy => '1', amopopr => '<(inet,inet)', @@ -2162,6 +2224,11 @@ amoprighttype => 'inet', amopstrategy => '5', amopopr => '>(inet,inet)', amopmethod => 'brin' }, +# bloom inet +{ amopfamily => 'brin/network_bloom_ops', amoplefttype => 'inet', + amoprighttype => 'inet', amopstrategy => '1', amopopr => '=(inet,inet)', + amopmethod => 'brin' }, + # inclusion inet { amopfamily => 'brin/network_inclusion_ops', amoplefttype => 'inet', amoprighttype => 'inet', amopstrategy => '3', amopopr => '&&(inet,inet)', @@ -2199,6 +2266,11 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# bloom character +{ amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, + # minmax time without time zone { amopfamily => 'brin/time_minmax_ops', amoplefttype => 'time', amoprighttype => 'time', amopstrategy => '1', amopopr => '<(time,time)', @@ -2216,6 +2288,11 @@ amoprighttype => 'time', amopstrategy => '5', amopopr => '>(time,time)', amopmethod => 'brin' }, +# bloom time without time zone +{ amopfamily => 'brin/time_bloom_ops', amoplefttype => 'time', + amoprighttype => 'time', amopstrategy => '1', amopopr => '=(time,time)', + amopmethod => 'brin' }, + # minmax datetime (date, timestamp, timestamptz) { amopfamily => 'brin/datetime_minmax_ops', amoplefttype => 'timestamp', @@ -2362,6 +2439,20 @@ amoprighttype => 'timestamptz', amopstrategy => '5', amopopr => '>(timestamptz,timestamptz)', amopmethod => 'brin' }, +# bloom datetime (date, timestamp, timestamptz) + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamp', + amoprighttype => 'timestamp', amopstrategy => '1', + amopopr => '=(timestamp,timestamp)', amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'date', + amoprighttype => 'date', amopstrategy => '1', amopopr => '=(date,date)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamptz', + amoprighttype => 'timestamptz', amopstrategy => '1', + amopopr => '=(timestamptz,timestamptz)', amopmethod => 'brin' }, + # minmax interval { amopfamily => 'brin/interval_minmax_ops', amoplefttype => 'interval', amoprighttype => 'interval', amopstrategy => '1', @@ -2379,6 +2470,11 @@ amoprighttype => 'interval', amopstrategy => '5', amopopr => '>(interval,interval)', amopmethod => 'brin' }, +# bloom interval +{ amopfamily => 'brin/interval_bloom_ops', amoplefttype => 'interval', + amoprighttype => 'interval', amopstrategy => '1', + amopopr => '=(interval,interval)', amopmethod => 'brin' }, + # minmax time with time zone { amopfamily => 'brin/timetz_minmax_ops', amoplefttype => 'timetz', amoprighttype => 'timetz', amopstrategy => '1', amopopr => '<(timetz,timetz)', @@ -2396,6 +2492,11 @@ amoprighttype => 'timetz', amopstrategy => '5', amopopr => '>(timetz,timetz)', amopmethod => 'brin' }, +# bloom time with time zone +{ amopfamily => 'brin/timetz_bloom_ops', amoplefttype => 'timetz', + amoprighttype => 'timetz', amopstrategy => '1', amopopr => '=(timetz,timetz)', + amopmethod => 'brin' }, + # minmax bit { amopfamily => 'brin/bit_minmax_ops', amoplefttype => 'bit', amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', @@ -2447,6 +2548,11 @@ amoprighttype => 'numeric', amopstrategy => '5', amopopr => '>(numeric,numeric)', amopmethod => 'brin' }, +# bloom numeric +{ amopfamily => 'brin/numeric_bloom_ops', amoplefttype => 'numeric', + amoprighttype => 'numeric', amopstrategy => '1', + amopopr => '=(numeric,numeric)', amopmethod => 'brin' }, + # minmax uuid { amopfamily => 'brin/uuid_minmax_ops', amoplefttype => 'uuid', amoprighttype => 'uuid', amopstrategy => '1', amopopr => '<(uuid,uuid)', @@ -2464,6 +2570,11 @@ amoprighttype => 'uuid', amopstrategy => '5', amopopr => '>(uuid,uuid)', amopmethod => 'brin' }, +# bloom uuid +{ amopfamily => 'brin/uuid_bloom_ops', amoplefttype => 'uuid', + amoprighttype => 'uuid', amopstrategy => '1', amopopr => '=(uuid,uuid)', + amopmethod => 'brin' }, + # inclusion range types { amopfamily => 'brin/range_inclusion_ops', amoplefttype => 'anyrange', amoprighttype => 'anyrange', amopstrategy => '1', @@ -2525,6 +2636,11 @@ amoprighttype => 'pg_lsn', amopstrategy => '5', amopopr => '>(pg_lsn,pg_lsn)', amopmethod => 'brin' }, +# bloom pg_lsn +{ amopfamily => 'brin/pg_lsn_bloom_ops', amoplefttype => 'pg_lsn', + amoprighttype => 'pg_lsn', amopstrategy => '1', amopopr => '=(pg_lsn,pg_lsn)', + amopmethod => 'brin' }, + # inclusion box { amopfamily => 'brin/box_inclusion_ops', amoplefttype => 'box', amoprighttype => 'box', amopstrategy => '1', amopopr => '<<(box,box)', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 36b5235c80..6709c8dfea 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -805,6 +805,24 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom bytea +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '11', amproc => 'hashvarlena' }, + # minmax "char" { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -818,6 +836,24 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom "char" +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '11', amproc => 'hashchar' }, + # minmax name { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -831,6 +867,24 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom name +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '11', amproc => 'hashname' }, + # minmax integer: int2, int4, int8 { amprocfamily => 'brin/integer_minmax_ops', amproclefttype => 'int8', amprocrighttype => 'int8', amprocnum => '1', @@ -932,6 +986,58 @@ { amprocfamily => 'brin/integer_minmax_ops', amproclefttype => 'int4', amprocrighttype => 'int2', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom integer: int2, int4, int8 +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '11', amproc => 'hashint8' }, + +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '11', amproc => 'hashint2' }, + +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '11', amproc => 'hashint4' }, + # minmax text { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -945,6 +1051,24 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom text +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '11', amproc => 'hashtext' }, + # minmax oid { amprocfamily => 'brin/oid_minmax_ops', amproclefttype => 'oid', amprocrighttype => 'oid', amprocnum => '1', amproc => 'brin_minmax_opcinfo' }, @@ -957,6 +1081,23 @@ { amprocfamily => 'brin/oid_minmax_ops', amproclefttype => 'oid', amprocrighttype => 'oid', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom oid +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '1', amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '11', amproc => 'hashoid' }, + # minmax tid { amprocfamily => 'brin/tid_minmax_ops', amproclefttype => 'tid', amprocrighttype => 'tid', amprocnum => '1', amproc => 'brin_minmax_opcinfo' }, @@ -969,6 +1110,23 @@ { amprocfamily => 'brin/tid_minmax_ops', amproclefttype => 'tid', amprocrighttype => 'tid', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom tid +{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid', + amprocrighttype => 'tid', amprocnum => '1', amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid', + amprocrighttype => 'tid', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid', + amprocrighttype => 'tid', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid', + amprocrighttype => 'tid', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid', + amprocrighttype => 'tid', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid', + amprocrighttype => 'tid', amprocnum => '11', amproc => 'hashtid' }, + # minmax float { amprocfamily => 'brin/float_minmax_ops', amproclefttype => 'float4', amprocrighttype => 'float4', amprocnum => '1', @@ -1019,6 +1177,45 @@ amprocrighttype => 'float4', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom float +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '11', + amproc => 'hashfloat4' }, + +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '11', + amproc => 'hashfloat8' }, + # minmax macaddr { amprocfamily => 'brin/macaddr_minmax_ops', amproclefttype => 'macaddr', amprocrighttype => 'macaddr', amprocnum => '1', @@ -1033,6 +1230,26 @@ amprocrighttype => 'macaddr', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom macaddr +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '11', + amproc => 'hashmacaddr' }, + # minmax macaddr8 { amprocfamily => 'brin/macaddr8_minmax_ops', amproclefttype => 'macaddr8', amprocrighttype => 'macaddr8', amprocnum => '1', @@ -1047,6 +1264,26 @@ amprocrighttype => 'macaddr8', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom macaddr8 +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '11', + amproc => 'hashmacaddr8' }, + # minmax inet { amprocfamily => 'brin/network_minmax_ops', amproclefttype => 'inet', amprocrighttype => 'inet', amprocnum => '1', @@ -1060,6 +1297,24 @@ { amprocfamily => 'brin/network_minmax_ops', amproclefttype => 'inet', amprocrighttype => 'inet', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom inet +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '11', amproc => 'hashinet' }, + # inclusion inet { amprocfamily => 'brin/network_inclusion_ops', amproclefttype => 'inet', amprocrighttype => 'inet', amprocnum => '1', @@ -1094,6 +1349,26 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom character +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '11', + amproc => 'hashchar' }, + # minmax time without time zone { amprocfamily => 'brin/time_minmax_ops', amproclefttype => 'time', amprocrighttype => 'time', amprocnum => '1', @@ -1107,6 +1382,24 @@ { amprocfamily => 'brin/time_minmax_ops', amproclefttype => 'time', amprocrighttype => 'time', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom time without time zone +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '11', amproc => 'time_hash' }, + # minmax datetime (date, timestamp, timestamptz) { amprocfamily => 'brin/datetime_minmax_ops', amproclefttype => 'timestamp', amprocrighttype => 'timestamp', amprocnum => '1', @@ -1214,6 +1507,62 @@ amprocrighttype => 'timestamptz', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom datetime (date, timestamp, timestamptz) +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '11', + amproc => 'timestamp_hash' }, + +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '11', + amproc => 'timestamp_hash' }, + +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '11', amproc => 'hashint4' }, + # minmax interval { amprocfamily => 'brin/interval_minmax_ops', amproclefttype => 'interval', amprocrighttype => 'interval', amprocnum => '1', @@ -1228,6 +1577,26 @@ amprocrighttype => 'interval', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom interval +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '11', + amproc => 'interval_hash' }, + # minmax time with time zone { amprocfamily => 'brin/timetz_minmax_ops', amproclefttype => 'timetz', amprocrighttype => 'timetz', amprocnum => '1', @@ -1242,6 +1611,26 @@ amprocrighttype => 'timetz', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom time with time zone +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '11', + amproc => 'timetz_hash' }, + # minmax bit { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_opcinfo' }, @@ -1282,6 +1671,26 @@ amprocrighttype => 'numeric', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom numeric +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '11', + amproc => 'hash_numeric' }, + # minmax uuid { amprocfamily => 'brin/uuid_minmax_ops', amproclefttype => 'uuid', amprocrighttype => 'uuid', amprocnum => '1', @@ -1295,6 +1704,24 @@ { amprocfamily => 'brin/uuid_minmax_ops', amproclefttype => 'uuid', amprocrighttype => 'uuid', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom uuid +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '11', amproc => 'uuid_hash' }, + # inclusion range types { amprocfamily => 'brin/range_inclusion_ops', amproclefttype => 'anyrange', amprocrighttype => 'anyrange', amprocnum => '1', @@ -1332,6 +1759,26 @@ amprocrighttype => 'pg_lsn', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom pg_lsn +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '11', + amproc => 'pg_lsn_hash' }, + # inclusion box { amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box', amprocrighttype => 'box', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index 24b1433e1f..6a5bb58baf 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -266,67 +266,130 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_bloom_ops', + opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_bloom_ops', + opcfamily => 'brin/char_bloom_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_bloom_ops', + opcfamily => 'brin/name_bloom_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'int8_minmax_ops', opcfamily => 'brin/integer_minmax_ops', opcintype => 'int8', opckeytype => 'int8' }, +{ opcmethod => 'brin', opcname => 'int8_bloom_ops', + opcfamily => 'brin/integer_bloom_ops', opcintype => 'int8', + opckeytype => 'int8', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'int2_minmax_ops', opcfamily => 'brin/integer_minmax_ops', opcintype => 'int2', opckeytype => 'int2' }, +{ opcmethod => 'brin', opcname => 'int2_bloom_ops', + opcfamily => 'brin/integer_bloom_ops', opcintype => 'int2', + opckeytype => 'int2', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'int4_minmax_ops', opcfamily => 'brin/integer_minmax_ops', opcintype => 'int4', opckeytype => 'int4' }, +{ opcmethod => 'brin', opcname => 'int4_bloom_ops', + opcfamily => 'brin/integer_bloom_ops', opcintype => 'int4', + opckeytype => 'int4', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_bloom_ops', + opcfamily => 'brin/text_bloom_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'oid_minmax_ops', opcfamily => 'brin/oid_minmax_ops', opcintype => 'oid', opckeytype => 'oid' }, +{ opcmethod => 'brin', opcname => 'oid_bloom_ops', + opcfamily => 'brin/oid_bloom_ops', opcintype => 'oid', + opckeytype => 'oid', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'tid_minmax_ops', opcfamily => 'brin/tid_minmax_ops', opcintype => 'tid', opckeytype => 'tid' }, +{ opcmethod => 'brin', opcname => 'tid_bloom_ops', + opcfamily => 'brin/tid_bloom_ops', opcintype => 'tid', opckeytype => 'tid', + opcdefault => 'f'}, { opcmethod => 'brin', opcname => 'float4_minmax_ops', opcfamily => 'brin/float_minmax_ops', opcintype => 'float4', opckeytype => 'float4' }, +{ opcmethod => 'brin', opcname => 'float4_bloom_ops', + opcfamily => 'brin/float_bloom_ops', opcintype => 'float4', + opckeytype => 'float4', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'float8_minmax_ops', opcfamily => 'brin/float_minmax_ops', opcintype => 'float8', opckeytype => 'float8' }, +{ opcmethod => 'brin', opcname => 'float8_bloom_ops', + opcfamily => 'brin/float_bloom_ops', opcintype => 'float8', + opckeytype => 'float8', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'macaddr_minmax_ops', opcfamily => 'brin/macaddr_minmax_ops', opcintype => 'macaddr', opckeytype => 'macaddr' }, +{ opcmethod => 'brin', opcname => 'macaddr_bloom_ops', + opcfamily => 'brin/macaddr_bloom_ops', opcintype => 'macaddr', + opckeytype => 'macaddr', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'macaddr8_minmax_ops', opcfamily => 'brin/macaddr8_minmax_ops', opcintype => 'macaddr8', opckeytype => 'macaddr8' }, +{ opcmethod => 'brin', opcname => 'macaddr8_bloom_ops', + opcfamily => 'brin/macaddr8_bloom_ops', opcintype => 'macaddr8', + opckeytype => 'macaddr8', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'inet_minmax_ops', opcfamily => 'brin/network_minmax_ops', opcintype => 'inet', opcdefault => 'f', opckeytype => 'inet' }, +{ opcmethod => 'brin', opcname => 'inet_bloom_ops', + opcfamily => 'brin/network_bloom_ops', opcintype => 'inet', + opcdefault => 'f', opckeytype => 'inet', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'inet_inclusion_ops', opcfamily => 'brin/network_inclusion_ops', opcintype => 'inet', opckeytype => 'inet' }, { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_bloom_ops', + opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'time_minmax_ops', opcfamily => 'brin/time_minmax_ops', opcintype => 'time', opckeytype => 'time' }, +{ opcmethod => 'brin', opcname => 'time_bloom_ops', + opcfamily => 'brin/time_bloom_ops', opcintype => 'time', + opckeytype => 'time', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'date_minmax_ops', opcfamily => 'brin/datetime_minmax_ops', opcintype => 'date', opckeytype => 'date' }, +{ opcmethod => 'brin', opcname => 'date_bloom_ops', + opcfamily => 'brin/datetime_bloom_ops', opcintype => 'date', + opckeytype => 'date', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'timestamp_minmax_ops', opcfamily => 'brin/datetime_minmax_ops', opcintype => 'timestamp', opckeytype => 'timestamp' }, +{ opcmethod => 'brin', opcname => 'timestamp_bloom_ops', + opcfamily => 'brin/datetime_bloom_ops', opcintype => 'timestamp', + opckeytype => 'timestamp', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'timestamptz_minmax_ops', opcfamily => 'brin/datetime_minmax_ops', opcintype => 'timestamptz', opckeytype => 'timestamptz' }, +{ opcmethod => 'brin', opcname => 'timestamptz_bloom_ops', + opcfamily => 'brin/datetime_bloom_ops', opcintype => 'timestamptz', + opckeytype => 'timestamptz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'interval_minmax_ops', opcfamily => 'brin/interval_minmax_ops', opcintype => 'interval', opckeytype => 'interval' }, +{ opcmethod => 'brin', opcname => 'interval_bloom_ops', + opcfamily => 'brin/interval_bloom_ops', opcintype => 'interval', + opckeytype => 'interval', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'timetz_minmax_ops', opcfamily => 'brin/timetz_minmax_ops', opcintype => 'timetz', opckeytype => 'timetz' }, +{ opcmethod => 'brin', opcname => 'timetz_bloom_ops', + opcfamily => 'brin/timetz_bloom_ops', opcintype => 'timetz', + opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', @@ -335,18 +398,27 @@ { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, +{ opcmethod => 'brin', opcname => 'numeric_bloom_ops', + opcfamily => 'brin/numeric_bloom_ops', opcintype => 'numeric', + opckeytype => 'numeric', opcdefault => 'f' }, # no brin opclass for record, anyarray { opcmethod => 'brin', opcname => 'uuid_minmax_ops', opcfamily => 'brin/uuid_minmax_ops', opcintype => 'uuid', opckeytype => 'uuid' }, +{ opcmethod => 'brin', opcname => 'uuid_bloom_ops', + opcfamily => 'brin/uuid_bloom_ops', opcintype => 'uuid', + opckeytype => 'uuid', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'range_inclusion_ops', opcfamily => 'brin/range_inclusion_ops', opcintype => 'anyrange', opckeytype => 'anyrange' }, { opcmethod => 'brin', opcname => 'pg_lsn_minmax_ops', opcfamily => 'brin/pg_lsn_minmax_ops', opcintype => 'pg_lsn', opckeytype => 'pg_lsn' }, +{ opcmethod => 'brin', opcname => 'pg_lsn_bloom_ops', + opcfamily => 'brin/pg_lsn_bloom_ops', opcintype => 'pg_lsn', + opckeytype => 'pg_lsn', opcdefault => 'f' }, # no brin opclass for enum, tsvector, tsquery, jsonb diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index 57e5aa0d8b..dea9adaf98 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -182,50 +182,88 @@ opfmethod => 'gin', opfname => 'jsonb_path_ops' }, { oid => '4054', opfmethod => 'brin', opfname => 'integer_minmax_ops' }, +{ oid => '9901', + opfmethod => 'brin', opfname => 'integer_bloom_ops' }, { oid => '4055', opfmethod => 'brin', opfname => 'numeric_minmax_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '9902', + opfmethod => 'brin', opfname => 'text_bloom_ops' }, +{ oid => '9903', + opfmethod => 'brin', opfname => 'numeric_bloom_ops' }, { oid => '4058', opfmethod => 'brin', opfname => 'timetz_minmax_ops' }, +{ oid => '9904', + opfmethod => 'brin', opfname => 'timetz_bloom_ops' }, { oid => '4059', opfmethod => 'brin', opfname => 'datetime_minmax_ops' }, +{ oid => '9905', + opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '9906', + opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '9907', + opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '9908', + opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', opfmethod => 'brin', opfname => 'oid_minmax_ops' }, +{ oid => '9909', + opfmethod => 'brin', opfname => 'oid_bloom_ops' }, { oid => '4069', opfmethod => 'brin', opfname => 'tid_minmax_ops' }, +{ oid => '9910', + opfmethod => 'brin', opfname => 'tid_bloom_ops' }, { oid => '4070', opfmethod => 'brin', opfname => 'float_minmax_ops' }, +{ oid => '9911', + opfmethod => 'brin', opfname => 'float_bloom_ops' }, { oid => '4074', opfmethod => 'brin', opfname => 'macaddr_minmax_ops' }, +{ oid => '9912', + opfmethod => 'brin', opfname => 'macaddr_bloom_ops' }, { oid => '4109', opfmethod => 'brin', opfname => 'macaddr8_minmax_ops' }, +{ oid => '9913', + opfmethod => 'brin', opfname => 'macaddr8_bloom_ops' }, { oid => '4075', opfmethod => 'brin', opfname => 'network_minmax_ops' }, { oid => '4102', opfmethod => 'brin', opfname => 'network_inclusion_ops' }, +{ oid => '9914', + opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '9915', + opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', opfmethod => 'brin', opfname => 'time_minmax_ops' }, +{ oid => '9916', + opfmethod => 'brin', opfname => 'time_bloom_ops' }, { oid => '4078', opfmethod => 'brin', opfname => 'interval_minmax_ops' }, +{ oid => '9917', + opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, +{ oid => '9918', + opfmethod => 'brin', opfname => 'uuid_bloom_ops' }, { oid => '4103', opfmethod => 'brin', opfname => 'range_inclusion_ops' }, { oid => '4082', opfmethod => 'brin', opfname => 'pg_lsn_minmax_ops' }, +{ oid => '9919', + opfmethod => 'brin', opfname => 'pg_lsn_bloom_ops' }, { oid => '4104', opfmethod => 'brin', opfname => 'box_inclusion_ops' }, { oid => '5000', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index ad11a2b66f..36c1433379 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8229,6 +8229,26 @@ proargtypes => 'internal internal internal', prosrc => 'brin_inclusion_union' }, +# BRIN bloom +{ oid => '9920', descr => 'BRIN bloom support', + proname => 'brin_bloom_opcinfo', prorettype => 'internal', + proargtypes => 'internal', prosrc => 'brin_bloom_opcinfo' }, +{ oid => '9921', descr => 'BRIN bloom support', + proname => 'brin_bloom_add_value', prorettype => 'bool', + proargtypes => 'internal internal internal internal', + prosrc => 'brin_bloom_add_value' }, +{ oid => '9922', descr => 'BRIN bloom support', + proname => 'brin_bloom_consistent', prorettype => 'bool', + proargtypes => 'internal internal internal int4', + prosrc => 'brin_bloom_consistent' }, +{ oid => '9923', descr => 'BRIN bloom support', + proname => 'brin_bloom_union', prorettype => 'bool', + proargtypes => 'internal internal internal', + prosrc => 'brin_bloom_union' }, +{ oid => '9924', descr => 'BRIN bloom support', + proname => 'brin_bloom_options', prorettype => 'void', proisstrict => 'f', + proargtypes => 'internal', prosrc => 'brin_bloom_options' }, + # userlock replacements { oid => '2880', descr => 'obtain exclusive advisory lock', proname => 'pg_advisory_lock', provolatile => 'v', proparallel => 'r', @@ -11402,4 +11422,18 @@ proname => 'is_normalized', prorettype => 'bool', proargtypes => 'text text', prosrc => 'unicode_is_normalized' }, +{ oid => '9035', descr => 'I/O', + proname => 'brin_bloom_summary_in', prorettype => 'pg_brin_bloom_summary', + proargtypes => 'cstring', prosrc => 'brin_bloom_summary_in' }, +{ oid => '9036', descr => 'I/O', + proname => 'brin_bloom_summary_out', prorettype => 'cstring', + proargtypes => 'pg_brin_bloom_summary', prosrc => 'brin_bloom_summary_out' }, +{ oid => '9037', descr => 'I/O', + proname => 'brin_bloom_summary_recv', provolatile => 's', + prorettype => 'pg_brin_bloom_summary', proargtypes => 'internal', + prosrc => 'brin_bloom_summary_recv' }, +{ oid => '9038', descr => 'I/O', + proname => 'brin_bloom_summary_send', provolatile => 's', prorettype => 'bytea', + proargtypes => 'pg_brin_bloom_summary', prosrc => 'brin_bloom_summary_send' }, + ] diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat index 8959c2f53b..74e279cbf9 100644 --- a/src/include/catalog/pg_type.dat +++ b/src/include/catalog/pg_type.dat @@ -679,5 +679,10 @@ typtype => 'p', typcategory => 'P', typinput => 'anycompatiblemultirange_in', typoutput => 'anycompatiblemultirange_out', typreceive => '-', typsend => '-', typalign => 'd', typstorage => 'x' }, - +{ oid => '9925', + descr => 'BRIN bloom summary', + typname => 'pg_brin_bloom_summary', typlen => '-1', typbyval => 'f', typcategory => 'S', + typinput => 'brin_bloom_summary_in', typoutput => 'brin_bloom_summary_out', + typreceive => 'brin_bloom_summary_recv', typsend => 'brin_bloom_summary_send', + typalign => 'i', typstorage => 'x', typcollation => 'default' }, ] diff --git a/src/test/regress/expected/brin_bloom.out b/src/test/regress/expected/brin_bloom.out new file mode 100644 index 0000000000..32c56a996a --- /dev/null +++ b/src/test/regress/expected/brin_bloom.out @@ -0,0 +1,428 @@ +CREATE TABLE brintest_bloom (byteacol bytea, + charcol "char", + namecol name, + int8col bigint, + int2col smallint, + int4col integer, + textcol text, + oidcol oid, + float4col real, + float8col double precision, + macaddrcol macaddr, + inetcol inet, + cidrcol cidr, + bpcharcol character, + datecol date, + timecol time without time zone, + timestampcol timestamp without time zone, + timestamptzcol timestamp with time zone, + intervalcol interval, + timetzcol time with time zone, + numericcol numeric, + uuidcol uuid, + lsncol pg_lsn +) WITH (fillfactor=10); +INSERT INTO brintest_bloom SELECT + repeat(stringu1, 8)::bytea, + substr(stringu1, 1, 1)::"char", + stringu1::name, 142857 * tenthous, + thousand, + twothousand, + repeat(stringu1, 8), + unique1::oid, + (four + 1.0)/(hundred+1), + odd::float8 / (tenthous + 1), + format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr, + inet '10.2.3.4/24' + tenthous, + cidr '10.2.3/24' + tenthous, + substr(stringu1, 1, 1)::bpchar, + date '1995-08-15' + tenthous, + time '01:20:30' + thousand * interval '18.5 second', + timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours', + timestamptz '1972-10-10 03:00' + thousand * interval '1 hour', + justify_days(justify_hours(tenthous * interval '12 minutes')), + timetz '01:30:20+02' + hundred * interval '15 seconds', + tenthous::numeric(36,30) * fivethous * even / (hundred + 1), + format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid, + format('%s/%s%s', odd, even, tenthous)::pg_lsn +FROM tenk1 ORDER BY unique2 LIMIT 100; +-- throw in some NULL's and different values +INSERT INTO brintest_bloom (inetcol, cidrcol) SELECT + inet 'fe80::6e40:8ff:fea9:8c46' + tenthous, + cidr 'fe80::6e40:8ff:fea9:8c46' + tenthous +FROM tenk1 ORDER BY thousand, tenthous LIMIT 25; +-- test bloom specific index options +-- ndistinct must be >= -1.0 +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(n_distinct_per_range = -1.1) +); +ERROR: value -1.1 out of bounds for option "n_distinct_per_range" +DETAIL: Valid values are between "-1.000000" and "2147483647.000000". +-- false_positive_rate must be between 0.0001 and 0.25 +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(false_positive_rate = 0.00009) +); +ERROR: value 0.00009 out of bounds for option "false_positive_rate" +DETAIL: Valid values are between "0.000100" and "0.250000". +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(false_positive_rate = 0.26) +); +ERROR: value 0.26 out of bounds for option "false_positive_rate" +DETAIL: Valid values are between "0.000100" and "0.250000". +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops, + charcol char_bloom_ops, + namecol name_bloom_ops, + int8col int8_bloom_ops, + int2col int2_bloom_ops, + int4col int4_bloom_ops, + textcol text_bloom_ops, + oidcol oid_bloom_ops, + float4col float4_bloom_ops, + float8col float8_bloom_ops, + macaddrcol macaddr_bloom_ops, + inetcol inet_bloom_ops, + cidrcol inet_bloom_ops, + bpcharcol bpchar_bloom_ops, + datecol date_bloom_ops, + timecol time_bloom_ops, + timestampcol timestamp_bloom_ops, + timestamptzcol timestamptz_bloom_ops, + intervalcol interval_bloom_ops, + timetzcol timetz_bloom_ops, + numericcol numeric_bloom_ops, + uuidcol uuid_bloom_ops, + lsncol pg_lsn_bloom_ops +) with (pages_per_range = 1); +CREATE TABLE brinopers_bloom (colname name, typ text, + op text[], value text[], matches int[], + check (cardinality(op) = cardinality(value)), + check (cardinality(op) = cardinality(matches))); +INSERT INTO brinopers_bloom VALUES + ('byteacol', 'bytea', + '{=}', + '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}', + '{1}'), + ('charcol', '"char"', + '{=}', + '{M}', + '{6}'), + ('namecol', 'name', + '{=}', + '{MAAAAA}', + '{2}'), + ('int2col', 'int2', + '{=}', + '{800}', + '{1}'), + ('int4col', 'int4', + '{=}', + '{800}', + '{1}'), + ('int8col', 'int8', + '{=}', + '{1257141600}', + '{1}'), + ('textcol', 'text', + '{=}', + '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}', + '{1}'), + ('oidcol', 'oid', + '{=}', + '{8800}', + '{1}'), + ('float4col', 'float4', + '{=}', + '{1}', + '{4}'), + ('float8col', 'float8', + '{=}', + '{0}', + '{1}'), + ('macaddrcol', 'macaddr', + '{=}', + '{2c:00:2d:00:16:00}', + '{2}'), + ('inetcol', 'inet', + '{=}', + '{10.2.14.231/24}', + '{1}'), + ('inetcol', 'cidr', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('cidrcol', 'inet', + '{=}', + '{10.2.14/24}', + '{2}'), + ('cidrcol', 'inet', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('cidrcol', 'cidr', + '{=}', + '{10.2.14/24}', + '{2}'), + ('cidrcol', 'cidr', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('bpcharcol', 'bpchar', + '{=}', + '{W}', + '{6}'), + ('datecol', 'date', + '{=}', + '{2009-12-01}', + '{1}'), + ('timecol', 'time', + '{=}', + '{02:28:57}', + '{1}'), + ('timestampcol', 'timestamp', + '{=}', + '{1964-03-24 19:26:45}', + '{1}'), + ('timestamptzcol', 'timestamptz', + '{=}', + '{1972-10-19 09:00:00-07}', + '{1}'), + ('intervalcol', 'interval', + '{=}', + '{1 mons 13 days 12:24}', + '{1}'), + ('timetzcol', 'timetz', + '{=}', + '{01:35:50+02}', + '{2}'), + ('numericcol', 'numeric', + '{=}', + '{2268164.347826086956521739130434782609}', + '{1}'), + ('uuidcol', 'uuid', + '{=}', + '{52225222-5222-5222-5222-522252225222}', + '{1}'), + ('lsncol', 'pg_lsn', + '{=, IS, IS NOT}', + '{44/455222, NULL, NULL}', + '{1, 25, 100}'); +DO $x$ +DECLARE + r record; + r2 record; + cond text; + idx_ctids tid[]; + ss_ctids tid[]; + count int; + plan_ok bool; + plan_line text; +BEGIN + FOR r IN SELECT colname, oper, typ, value[ordinality], matches[ordinality] FROM brinopers_bloom, unnest(op) WITH ORDINALITY AS oper LOOP + + -- prepare the condition + IF r.value IS NULL THEN + cond := format('%I %s %L', r.colname, r.oper, r.value); + ELSE + cond := format('%I %s %L::%s', r.colname, r.oper, r.value, r.typ); + END IF; + + -- run the query using the brin index + SET enable_seqscan = 0; + SET enable_bitmapscan = 1; + + plan_ok := false; + FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP + IF plan_line LIKE '%Bitmap Heap Scan on brintest_bloom%' THEN + plan_ok := true; + END IF; + END LOOP; + IF NOT plan_ok THEN + RAISE WARNING 'did not get bitmap indexscan plan for %', r; + END IF; + + EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) + INTO idx_ctids; + + -- run the query using a seqscan + SET enable_seqscan = 1; + SET enable_bitmapscan = 0; + + plan_ok := false; + FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP + IF plan_line LIKE '%Seq Scan on brintest_bloom%' THEN + plan_ok := true; + END IF; + END LOOP; + IF NOT plan_ok THEN + RAISE WARNING 'did not get seqscan plan for %', r; + END IF; + + EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) + INTO ss_ctids; + + -- make sure both return the same results + count := array_length(idx_ctids, 1); + + IF NOT (count = array_length(ss_ctids, 1) AND + idx_ctids @> ss_ctids AND + idx_ctids <@ ss_ctids) THEN + -- report the results of each scan to make the differences obvious + RAISE WARNING 'something not right in %: count %', r, count; + SET enable_seqscan = 1; + SET enable_bitmapscan = 0; + FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP + RAISE NOTICE 'seqscan: %', r2; + END LOOP; + + SET enable_seqscan = 0; + SET enable_bitmapscan = 1; + FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP + RAISE NOTICE 'bitmapscan: %', r2; + END LOOP; + END IF; + + -- make sure we found expected number of matches + IF count != r.matches THEN RAISE WARNING 'unexpected number of results % for %', count, r; END IF; + END LOOP; +END; +$x$; +RESET enable_seqscan; +RESET enable_bitmapscan; +INSERT INTO brintest_bloom SELECT + repeat(stringu1, 42)::bytea, + substr(stringu1, 1, 1)::"char", + stringu1::name, 142857 * tenthous, + thousand, + twothousand, + repeat(stringu1, 42), + unique1::oid, + (four + 1.0)/(hundred+1), + odd::float8 / (tenthous + 1), + format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr, + inet '10.2.3.4' + tenthous, + cidr '10.2.3/24' + tenthous, + substr(stringu1, 1, 1)::bpchar, + date '1995-08-15' + tenthous, + time '01:20:30' + thousand * interval '18.5 second', + timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours', + timestamptz '1972-10-10 03:00' + thousand * interval '1 hour', + justify_days(justify_hours(tenthous * interval '12 minutes')), + timetz '01:30:20' + hundred * interval '15 seconds', + tenthous::numeric(36,30) * fivethous * even / (hundred + 1), + format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid, + format('%s/%s%s', odd, even, tenthous)::pg_lsn +FROM tenk1 ORDER BY unique2 LIMIT 5 OFFSET 5; +SELECT brin_desummarize_range('brinidx_bloom', 0); + brin_desummarize_range +------------------------ + +(1 row) + +VACUUM brintest_bloom; -- force a summarization cycle in brinidx +UPDATE brintest_bloom SET int8col = int8col * int4col; +UPDATE brintest_bloom SET textcol = '' WHERE textcol IS NOT NULL; +-- Tests for brin_summarize_new_values +SELECT brin_summarize_new_values('brintest_bloom'); -- error, not an index +ERROR: "brintest_bloom" is not an index +SELECT brin_summarize_new_values('tenk1_unique1'); -- error, not a BRIN index +ERROR: "tenk1_unique1" is not a BRIN index +SELECT brin_summarize_new_values('brinidx_bloom'); -- ok, no change expected + brin_summarize_new_values +--------------------------- + 0 +(1 row) + +-- Tests for brin_desummarize_range +SELECT brin_desummarize_range('brinidx_bloom', -1); -- error, invalid range +ERROR: block number out of range: -1 +SELECT brin_desummarize_range('brinidx_bloom', 0); + brin_desummarize_range +------------------------ + +(1 row) + +SELECT brin_desummarize_range('brinidx_bloom', 0); + brin_desummarize_range +------------------------ + +(1 row) + +SELECT brin_desummarize_range('brinidx_bloom', 100000000); + brin_desummarize_range +------------------------ + +(1 row) + +-- Test brin_summarize_range +CREATE TABLE brin_summarize_bloom ( + value int +) WITH (fillfactor=10, autovacuum_enabled=false); +CREATE INDEX brin_summarize_bloom_idx ON brin_summarize_bloom USING brin (value) WITH (pages_per_range=2); +-- Fill a few pages +DO $$ +DECLARE curtid tid; +BEGIN + LOOP + INSERT INTO brin_summarize_bloom VALUES (1) RETURNING ctid INTO curtid; + EXIT WHEN curtid > tid '(2, 0)'; + END LOOP; +END; +$$; +-- summarize one range +SELECT brin_summarize_range('brin_summarize_bloom_idx', 0); + brin_summarize_range +---------------------- + 0 +(1 row) + +-- nothing: already summarized +SELECT brin_summarize_range('brin_summarize_bloom_idx', 1); + brin_summarize_range +---------------------- + 0 +(1 row) + +-- summarize one range +SELECT brin_summarize_range('brin_summarize_bloom_idx', 2); + brin_summarize_range +---------------------- + 1 +(1 row) + +-- nothing: page doesn't exist in table +SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967295); + brin_summarize_range +---------------------- + 0 +(1 row) + +-- invalid block number values +SELECT brin_summarize_range('brin_summarize_bloom_idx', -1); +ERROR: block number out of range: -1 +SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967296); +ERROR: block number out of range: 4294967296 +-- test brin cost estimates behave sanely based on correlation of values +CREATE TABLE brin_test_bloom (a INT, b INT); +INSERT INTO brin_test_bloom SELECT x/100,x%100 FROM generate_series(1,10000) x(x); +CREATE INDEX brin_test_bloom_a_idx ON brin_test_bloom USING brin (a) WITH (pages_per_range = 2); +CREATE INDEX brin_test_bloom_b_idx ON brin_test_bloom USING brin (b) WITH (pages_per_range = 2); +VACUUM ANALYZE brin_test_bloom; +-- Ensure brin index is used when columns are perfectly correlated +EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE a = 1; + QUERY PLAN +-------------------------------------------------- + Bitmap Heap Scan on brin_test_bloom + Recheck Cond: (a = 1) + -> Bitmap Index Scan on brin_test_bloom_a_idx + Index Cond: (a = 1) +(4 rows) + +-- Ensure brin index is not used when values are not correlated +EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE b = 1; + QUERY PLAN +----------------------------- + Seq Scan on brin_test_bloom + Filter: (b = 1) +(2 rows) + diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..ef4b4444b9 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -2037,6 +2037,7 @@ ORDER BY 1, 2, 3; 2742 | 16 | @@ 3580 | 1 | < 3580 | 1 | << + 3580 | 1 | = 3580 | 2 | &< 3580 | 2 | <= 3580 | 3 | && @@ -2100,7 +2101,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out index 7204fdb0b4..a7a5b22d4f 100644 --- a/src/test/regress/expected/psql.out +++ b/src/test/regress/expected/psql.out @@ -4993,8 +4993,9 @@ List of access methods List of operator classes AM | Input type | Storage type | Operator class | Default? ------+------------+--------------+----------------+---------- + brin | oid | | oid_bloom_ops | no brin | oid | | oid_minmax_ops | yes -(1 row) +(2 rows) \dAf spgist List of operator families diff --git a/src/test/regress/expected/type_sanity.out b/src/test/regress/expected/type_sanity.out index 0c74dc96a8..e568b9fea2 100644 --- a/src/test/regress/expected/type_sanity.out +++ b/src/test/regress/expected/type_sanity.out @@ -67,13 +67,14 @@ WHERE p1.typtype not in ('p') AND p1.typname NOT LIKE E'\\_%' WHERE p2.typname = ('_' || p1.typname)::name AND p2.typelem = p1.oid and p1.typarray = p2.oid) ORDER BY p1.oid; - oid | typname -------+----------------- + oid | typname +------+----------------------- 194 | pg_node_tree 3361 | pg_ndistinct 3402 | pg_dependencies 5017 | pg_mcv_list -(4 rows) + 9925 | pg_brin_bloom_summary +(5 rows) -- Make sure typarray points to a "true" array type of our own base SELECT p1.oid, p1.typname as basetype, p2.typname as arraytype, diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index c77b0d7342..ecd0806718 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -77,6 +77,11 @@ test: select_into select_distinct select_distinct_on select_implicit select_havi # ---------- test: brin gin gist spgist privileges init_privs security_label collate matview lock replica_identity rowsecurity object_address tablesample groupingsets drop_operator password identity generated join_hash +# ---------- +# Additional BRIN tests +# ---------- +test: brin_bloom + # ---------- # Another group of parallel tests # ---------- diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule index 0264a97324..c0d7fa76f1 100644 --- a/src/test/regress/serial_schedule +++ b/src/test/regress/serial_schedule @@ -108,6 +108,7 @@ test: delete test: namespace test: prepared_xacts test: brin +test: brin_bloom test: gin test: gist test: spgist diff --git a/src/test/regress/sql/brin_bloom.sql b/src/test/regress/sql/brin_bloom.sql new file mode 100644 index 0000000000..5d499208e3 --- /dev/null +++ b/src/test/regress/sql/brin_bloom.sql @@ -0,0 +1,376 @@ +CREATE TABLE brintest_bloom (byteacol bytea, + charcol "char", + namecol name, + int8col bigint, + int2col smallint, + int4col integer, + textcol text, + oidcol oid, + float4col real, + float8col double precision, + macaddrcol macaddr, + inetcol inet, + cidrcol cidr, + bpcharcol character, + datecol date, + timecol time without time zone, + timestampcol timestamp without time zone, + timestamptzcol timestamp with time zone, + intervalcol interval, + timetzcol time with time zone, + numericcol numeric, + uuidcol uuid, + lsncol pg_lsn +) WITH (fillfactor=10); + +INSERT INTO brintest_bloom SELECT + repeat(stringu1, 8)::bytea, + substr(stringu1, 1, 1)::"char", + stringu1::name, 142857 * tenthous, + thousand, + twothousand, + repeat(stringu1, 8), + unique1::oid, + (four + 1.0)/(hundred+1), + odd::float8 / (tenthous + 1), + format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr, + inet '10.2.3.4/24' + tenthous, + cidr '10.2.3/24' + tenthous, + substr(stringu1, 1, 1)::bpchar, + date '1995-08-15' + tenthous, + time '01:20:30' + thousand * interval '18.5 second', + timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours', + timestamptz '1972-10-10 03:00' + thousand * interval '1 hour', + justify_days(justify_hours(tenthous * interval '12 minutes')), + timetz '01:30:20+02' + hundred * interval '15 seconds', + tenthous::numeric(36,30) * fivethous * even / (hundred + 1), + format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid, + format('%s/%s%s', odd, even, tenthous)::pg_lsn +FROM tenk1 ORDER BY unique2 LIMIT 100; + +-- throw in some NULL's and different values +INSERT INTO brintest_bloom (inetcol, cidrcol) SELECT + inet 'fe80::6e40:8ff:fea9:8c46' + tenthous, + cidr 'fe80::6e40:8ff:fea9:8c46' + tenthous +FROM tenk1 ORDER BY thousand, tenthous LIMIT 25; + +-- test bloom specific index options +-- ndistinct must be >= -1.0 +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(n_distinct_per_range = -1.1) +); +-- false_positive_rate must be between 0.0001 and 0.25 +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(false_positive_rate = 0.00009) +); +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(false_positive_rate = 0.26) +); + +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops, + charcol char_bloom_ops, + namecol name_bloom_ops, + int8col int8_bloom_ops, + int2col int2_bloom_ops, + int4col int4_bloom_ops, + textcol text_bloom_ops, + oidcol oid_bloom_ops, + float4col float4_bloom_ops, + float8col float8_bloom_ops, + macaddrcol macaddr_bloom_ops, + inetcol inet_bloom_ops, + cidrcol inet_bloom_ops, + bpcharcol bpchar_bloom_ops, + datecol date_bloom_ops, + timecol time_bloom_ops, + timestampcol timestamp_bloom_ops, + timestamptzcol timestamptz_bloom_ops, + intervalcol interval_bloom_ops, + timetzcol timetz_bloom_ops, + numericcol numeric_bloom_ops, + uuidcol uuid_bloom_ops, + lsncol pg_lsn_bloom_ops +) with (pages_per_range = 1); + +CREATE TABLE brinopers_bloom (colname name, typ text, + op text[], value text[], matches int[], + check (cardinality(op) = cardinality(value)), + check (cardinality(op) = cardinality(matches))); + +INSERT INTO brinopers_bloom VALUES + ('byteacol', 'bytea', + '{=}', + '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}', + '{1}'), + ('charcol', '"char"', + '{=}', + '{M}', + '{6}'), + ('namecol', 'name', + '{=}', + '{MAAAAA}', + '{2}'), + ('int2col', 'int2', + '{=}', + '{800}', + '{1}'), + ('int4col', 'int4', + '{=}', + '{800}', + '{1}'), + ('int8col', 'int8', + '{=}', + '{1257141600}', + '{1}'), + ('textcol', 'text', + '{=}', + '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}', + '{1}'), + ('oidcol', 'oid', + '{=}', + '{8800}', + '{1}'), + ('float4col', 'float4', + '{=}', + '{1}', + '{4}'), + ('float8col', 'float8', + '{=}', + '{0}', + '{1}'), + ('macaddrcol', 'macaddr', + '{=}', + '{2c:00:2d:00:16:00}', + '{2}'), + ('inetcol', 'inet', + '{=}', + '{10.2.14.231/24}', + '{1}'), + ('inetcol', 'cidr', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('cidrcol', 'inet', + '{=}', + '{10.2.14/24}', + '{2}'), + ('cidrcol', 'inet', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('cidrcol', 'cidr', + '{=}', + '{10.2.14/24}', + '{2}'), + ('cidrcol', 'cidr', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('bpcharcol', 'bpchar', + '{=}', + '{W}', + '{6}'), + ('datecol', 'date', + '{=}', + '{2009-12-01}', + '{1}'), + ('timecol', 'time', + '{=}', + '{02:28:57}', + '{1}'), + ('timestampcol', 'timestamp', + '{=}', + '{1964-03-24 19:26:45}', + '{1}'), + ('timestamptzcol', 'timestamptz', + '{=}', + '{1972-10-19 09:00:00-07}', + '{1}'), + ('intervalcol', 'interval', + '{=}', + '{1 mons 13 days 12:24}', + '{1}'), + ('timetzcol', 'timetz', + '{=}', + '{01:35:50+02}', + '{2}'), + ('numericcol', 'numeric', + '{=}', + '{2268164.347826086956521739130434782609}', + '{1}'), + ('uuidcol', 'uuid', + '{=}', + '{52225222-5222-5222-5222-522252225222}', + '{1}'), + ('lsncol', 'pg_lsn', + '{=, IS, IS NOT}', + '{44/455222, NULL, NULL}', + '{1, 25, 100}'); + +DO $x$ +DECLARE + r record; + r2 record; + cond text; + idx_ctids tid[]; + ss_ctids tid[]; + count int; + plan_ok bool; + plan_line text; +BEGIN + FOR r IN SELECT colname, oper, typ, value[ordinality], matches[ordinality] FROM brinopers_bloom, unnest(op) WITH ORDINALITY AS oper LOOP + + -- prepare the condition + IF r.value IS NULL THEN + cond := format('%I %s %L', r.colname, r.oper, r.value); + ELSE + cond := format('%I %s %L::%s', r.colname, r.oper, r.value, r.typ); + END IF; + + -- run the query using the brin index + SET enable_seqscan = 0; + SET enable_bitmapscan = 1; + + plan_ok := false; + FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP + IF plan_line LIKE '%Bitmap Heap Scan on brintest_bloom%' THEN + plan_ok := true; + END IF; + END LOOP; + IF NOT plan_ok THEN + RAISE WARNING 'did not get bitmap indexscan plan for %', r; + END IF; + + EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) + INTO idx_ctids; + + -- run the query using a seqscan + SET enable_seqscan = 1; + SET enable_bitmapscan = 0; + + plan_ok := false; + FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP + IF plan_line LIKE '%Seq Scan on brintest_bloom%' THEN + plan_ok := true; + END IF; + END LOOP; + IF NOT plan_ok THEN + RAISE WARNING 'did not get seqscan plan for %', r; + END IF; + + EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) + INTO ss_ctids; + + -- make sure both return the same results + count := array_length(idx_ctids, 1); + + IF NOT (count = array_length(ss_ctids, 1) AND + idx_ctids @> ss_ctids AND + idx_ctids <@ ss_ctids) THEN + -- report the results of each scan to make the differences obvious + RAISE WARNING 'something not right in %: count %', r, count; + SET enable_seqscan = 1; + SET enable_bitmapscan = 0; + FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP + RAISE NOTICE 'seqscan: %', r2; + END LOOP; + + SET enable_seqscan = 0; + SET enable_bitmapscan = 1; + FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP + RAISE NOTICE 'bitmapscan: %', r2; + END LOOP; + END IF; + + -- make sure we found expected number of matches + IF count != r.matches THEN RAISE WARNING 'unexpected number of results % for %', count, r; END IF; + END LOOP; +END; +$x$; + +RESET enable_seqscan; +RESET enable_bitmapscan; + +INSERT INTO brintest_bloom SELECT + repeat(stringu1, 42)::bytea, + substr(stringu1, 1, 1)::"char", + stringu1::name, 142857 * tenthous, + thousand, + twothousand, + repeat(stringu1, 42), + unique1::oid, + (four + 1.0)/(hundred+1), + odd::float8 / (tenthous + 1), + format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr, + inet '10.2.3.4' + tenthous, + cidr '10.2.3/24' + tenthous, + substr(stringu1, 1, 1)::bpchar, + date '1995-08-15' + tenthous, + time '01:20:30' + thousand * interval '18.5 second', + timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours', + timestamptz '1972-10-10 03:00' + thousand * interval '1 hour', + justify_days(justify_hours(tenthous * interval '12 minutes')), + timetz '01:30:20' + hundred * interval '15 seconds', + tenthous::numeric(36,30) * fivethous * even / (hundred + 1), + format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid, + format('%s/%s%s', odd, even, tenthous)::pg_lsn +FROM tenk1 ORDER BY unique2 LIMIT 5 OFFSET 5; + +SELECT brin_desummarize_range('brinidx_bloom', 0); +VACUUM brintest_bloom; -- force a summarization cycle in brinidx + +UPDATE brintest_bloom SET int8col = int8col * int4col; +UPDATE brintest_bloom SET textcol = '' WHERE textcol IS NOT NULL; + +-- Tests for brin_summarize_new_values +SELECT brin_summarize_new_values('brintest_bloom'); -- error, not an index +SELECT brin_summarize_new_values('tenk1_unique1'); -- error, not a BRIN index +SELECT brin_summarize_new_values('brinidx_bloom'); -- ok, no change expected + +-- Tests for brin_desummarize_range +SELECT brin_desummarize_range('brinidx_bloom', -1); -- error, invalid range +SELECT brin_desummarize_range('brinidx_bloom', 0); +SELECT brin_desummarize_range('brinidx_bloom', 0); +SELECT brin_desummarize_range('brinidx_bloom', 100000000); + +-- Test brin_summarize_range +CREATE TABLE brin_summarize_bloom ( + value int +) WITH (fillfactor=10, autovacuum_enabled=false); +CREATE INDEX brin_summarize_bloom_idx ON brin_summarize_bloom USING brin (value) WITH (pages_per_range=2); +-- Fill a few pages +DO $$ +DECLARE curtid tid; +BEGIN + LOOP + INSERT INTO brin_summarize_bloom VALUES (1) RETURNING ctid INTO curtid; + EXIT WHEN curtid > tid '(2, 0)'; + END LOOP; +END; +$$; + +-- summarize one range +SELECT brin_summarize_range('brin_summarize_bloom_idx', 0); +-- nothing: already summarized +SELECT brin_summarize_range('brin_summarize_bloom_idx', 1); +-- summarize one range +SELECT brin_summarize_range('brin_summarize_bloom_idx', 2); +-- nothing: page doesn't exist in table +SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967295); +-- invalid block number values +SELECT brin_summarize_range('brin_summarize_bloom_idx', -1); +SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967296); + + +-- test brin cost estimates behave sanely based on correlation of values +CREATE TABLE brin_test_bloom (a INT, b INT); +INSERT INTO brin_test_bloom SELECT x/100,x%100 FROM generate_series(1,10000) x(x); +CREATE INDEX brin_test_bloom_a_idx ON brin_test_bloom USING brin (a) WITH (pages_per_range = 2); +CREATE INDEX brin_test_bloom_b_idx ON brin_test_bloom USING brin (b) WITH (pages_per_range = 2); +VACUUM ANALYZE brin_test_bloom; + +-- Ensure brin index is used when columns are perfectly correlated +EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE a = 1; +-- Ensure brin index is not used when values are not correlated +EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE b = 1; -- 2.26.2 --------------556A1DC61262AF15641DB204 Content-Type: text/x-patch; charset=UTF-8; name="0007-BRIN-minmax-multi-indexes-20210308b.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0007-BRIN-minmax-multi-indexes-20210308b.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-07 12:43 jian he <[email protected]> 0 siblings, 1 reply; 11+ messages in thread From: jian he @ 2025-04-07 12:43 UTC (permalink / raw) To: Rushabh Lathia <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; pgsql-hackers hi. CREATE TABLE t (a int, b int); INSERT INTO t VALUES (NULL, 1), (300, 3); ALTER TABLE t ADD CONSTRAINT nn NOT NULL a NOT VALID; -- ok ALTER TABLE t add column c float8 default random(); the last query should not fail. if we want more places use CompactAttribute->attnullability set_attnotnull should also set CompactAttribute->attnullability proactively not waiting CommandCounterIncrement invoke RelationBuildTupleDesc. another reason: CheckNNConstraintFetch only handle "if (!conform->convalidated)" what if the not-null is set conform->convalidated to true, then RelationBuildTupleDesc doesn't set/change CompactAttribute->attnullability at all. attached is all the needed changes after v8-0001, v8-0002, i think. Attachments: [application/octet-stream] v8-0003-not-null-not-valid-miscellous-fix.no-cfbot (5.6K, ../../CACJufxGC7pZsJFBkU1mUG8vdGGoRQh8otiZkA0aC7bULktBosw@mail.gmail.com/2-v8-0003-not-null-not-valid-miscellous-fix.no-cfbot) download ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-07 17:24 Alvaro Herrera <[email protected]> parent: jian he <[email protected]> 0 siblings, 2 replies; 11+ messages in thread From: Alvaro Herrera @ 2025-04-07 17:24 UTC (permalink / raw) To: jian he <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; pgsql-hackers On 2025-Apr-07, jian he wrote: > CREATE TABLE t (a int, b int); > INSERT INTO t VALUES (NULL, 1), (300, 3); > ALTER TABLE t ADD CONSTRAINT nn NOT NULL a NOT VALID; -- ok > ALTER TABLE t add column c float8 default random(); > the last query should not fail. Agreed. > if we want more places use CompactAttribute->attnullability > set_attnotnull should also set CompactAttribute->attnullability proactively > not waiting CommandCounterIncrement invoke RelationBuildTupleDesc. Actually, the fix here was to tweak equalTupleDescs to also compare attnullability, and then ensure that a relcache invalidation happens. That way the old tupdesc goes away correctly. I have pushed this after some small additional changes. I ran some of the tests under debug_discard_caches=1 to make sure that invalidations are handled correctly also. I have to admit that this patch was much more difficult than I had initially anticipated. Thank you Rushabh very much for the effort in writing and rewriting as the different ideas came and went, and Jian for the eagle eyes and the additional test cases and debugging. Cheers -- Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/ "In Europe they call me Niklaus Wirth; in the US they call me Nickel's worth. That's because in Europe they call me by name, and in the US by value!" ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-07 18:30 Tom Lane <[email protected]> parent: Alvaro Herrera <[email protected]> 1 sibling, 0 replies; 11+ messages in thread From: Tom Lane @ 2025-04-07 18:30 UTC (permalink / raw) To: Alvaro Herrera <[email protected]>; +Cc: jian he <[email protected]>; Rushabh Lathia <[email protected]>; pgsql-hackers Alvaro Herrera <[email protected]> writes: > I have pushed this after some small additional changes. Looks like some of the test cases have issues with locale-dependent ordering. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jay&dt=2025-04-07%2017%3A56%3A49 regards, tom lane ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-08 00:21 Rushabh Lathia <[email protected]> parent: Alvaro Herrera <[email protected]> 1 sibling, 1 reply; 11+ messages in thread From: Rushabh Lathia @ 2025-04-08 00:21 UTC (permalink / raw) To: Alvaro Herrera <[email protected]>; +Cc: jian he <[email protected]>; pgsql-hackers On Tue, Apr 8, 2025 at 1:24 AM Alvaro Herrera <[email protected]> wrote: > On 2025-Apr-07, jian he wrote: > > > CREATE TABLE t (a int, b int); > > INSERT INTO t VALUES (NULL, 1), (300, 3); > > ALTER TABLE t ADD CONSTRAINT nn NOT NULL a NOT VALID; -- ok > > ALTER TABLE t add column c float8 default random(); > > the last query should not fail. > > Agreed. > > > if we want more places use CompactAttribute->attnullability > > set_attnotnull should also set CompactAttribute->attnullability > proactively > > not waiting CommandCounterIncrement invoke RelationBuildTupleDesc. > > Actually, the fix here was to tweak equalTupleDescs to also compare > attnullability, and then ensure that a relcache invalidation happens. > That way the old tupdesc goes away correctly. > Ah okay. > > I have pushed this after some small additional changes. I ran some of > the tests under debug_discard_caches=1 to make sure that invalidations > are handled correctly also. > > I have to admit that this patch was much more difficult than I had > initially anticipated. Thank you Rushabh very much for the effort in > writing and rewriting as the different ideas came and went, and Jian for > the eagle eyes and the additional test cases and debugging. > Thank you Alvaro for your support and guidance. Thanks Jian. > Cheers > > -- > Álvaro Herrera Breisgau, Deutschland — > https://www.EnterpriseDB.com/ > "In Europe they call me Niklaus Wirth; in the US they call me Nickel's > worth. > That's because in Europe they call me by name, and in the US by value!" > -- Rushabh Lathia ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-09 05:16 jian he <[email protected]> parent: Rushabh Lathia <[email protected]> 0 siblings, 2 replies; 11+ messages in thread From: jian he @ 2025-04-09 05:16 UTC (permalink / raw) To: Rushabh Lathia <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; pgsql-hackers hi. attached patch is for address pg_dump inconsistency when parent is "not null not valid" while child is "not null". The following query before/after pg_dump should return the same result. select conrelid::regclass::text, conname, convalidated, coninhcount, conislocal, conparentid, contype from pg_constraint where conrelid::regclass::text = ANY('{inhnn, inhnn_cc, inhnn_cc_1}') order by 1,2; --test cases: CREATE TABLE inhnn (a INTEGER); ALTER TABLE inhnn ADD CONSTRAINT cc not null a NOT VALID; CREATE TABLE inhnn_cc(a INTEGER) INHERITS(inhnn); CREATE TABLE inhnn_cc_1(a INTEGER) INHERITS(inhnn_cc, inhnn); master pg_dump output is: CREATE TABLE public.inhnn (a integer); CREATE TABLE public.inhnn_cc (a integer) INHERITS (public.inhnn); CREATE TABLE public.inhnn_cc_1 (a integer) INHERITS (public.inhnn_cc, public.inhnn); ALTER TABLE public.inhnn ADD CONSTRAINT cc NOT NULL a NOT VALID; with the attached patch, pg_dump output is: CREATE TABLE public.inhnn (a integer); CREATE TABLE public.inhnn_cc (a integer CONSTRAINT cc NOT NULL) INHERITS (public.inhnn); CREATE TABLE public.inhnn_cc_1 (a integer CONSTRAINT cc NOT NULL) INHERITS (public.inhnn_cc, public.inhnn); ALTER TABLE public.inhnn ADD CONSTRAINT cc NOT NULL a NOT VALID; ------------- As you can see, in master, pg_dump will make {inhnn, inhnn_cc, inhnn_cc_1} not-null constraint's pg_constraint.convalidated set as false. but we should only make inhnn's not-null constraint convalidated as false. Attachments: [text/x-patch] v1-0001-pg_dump-not-null-not-valid.patch (5.1K, ../../CACJufxGHNNMc0E2JphUqJMzD3=bwRSuAEVBF5ekgkG8uY0Q3hg@mail.gmail.com/2-v1-0001-pg_dump-not-null-not-valid.patch) download | inline diff: From 9505f36287403aa8efd7642dddf71b77996796dd Mon Sep 17 00:00:00 2001 From: jian he <[email protected]> Date: Wed, 9 Apr 2025 13:07:58 +0800 Subject: [PATCH v1 1/1] pg_dump not null not valid make sure pg_dump have the same pg_constraint meta before and after pg_dump. if the parent not-null constraint is invalid, child is valid. then pg_dump need locally print the not-null constraint on that child too. otherwise pg_dump may make child's convalidated may set to false. that means we also need adjust conislocal in AdjustNotNullInheritance. --- src/backend/catalog/pg_constraint.c | 10 ++++++++++ src/bin/pg_dump/common.c | 4 ++++ src/bin/pg_dump/pg_dump.c | 17 +++++++++++++++++ src/bin/pg_dump/pg_dump.h | 5 +++++ 4 files changed, 36 insertions(+) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index 2f73085961b..9e65b96143f 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -776,6 +776,16 @@ AdjustNotNullInheritance(Oid relid, AttrNumber attnum, ereport(ERROR, errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), errmsg("too many inheritance parents")); + + /* + * If the child already has a valid constraint and we are + * creating an invalid one with same definition on it. The + * child's constraint will remain valid, but can no longer be + * marked as local. + */ + if (is_notvalid && conform->convalidated && conform->conenforced) + conform->conislocal = false; + changed = true; } else if (!conform->conislocal) diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c index 56b6c368acf..ff6a4eacda0 100644 --- a/src/bin/pg_dump/common.c +++ b/src/bin/pg_dump/common.c @@ -546,6 +546,10 @@ flagInhAttrs(Archive *fout, DumpOptions *dopt, TableInfo *tblinfo, int numTables parent->notnull_constrs[inhAttrInd] != NULL) foundNotNull = true; + if (fout->remoteVersion >= 180000 && + parent->notnull_invalid[inhAttrInd]) + tbinfo->notnull_parent_invalid[j] = true; + foundDefault |= (parentDef != NULL && strcmp(parentDef->adef_expr, "NULL") != 0 && !parent->attgenerated[inhAttrInd]); diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index 25264f8c9fb..8d131523366 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -9255,6 +9255,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) tbinfo->attfdwoptions = (char **) pg_malloc(numatts * sizeof(char *)); tbinfo->attmissingval = (char **) pg_malloc(numatts * sizeof(char *)); tbinfo->notnull_constrs = (char **) pg_malloc(numatts * sizeof(char *)); + tbinfo->notnull_invalid = (bool *) pg_malloc(numatts * sizeof(bool)); + tbinfo->notnull_parent_invalid = (bool *) pg_malloc(numatts * sizeof(bool)); tbinfo->notnull_noinh = (bool *) pg_malloc(numatts * sizeof(bool)); tbinfo->notnull_islocal = (bool *) pg_malloc(numatts * sizeof(bool)); tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(numatts * sizeof(AttrDefInfo *)); @@ -9280,6 +9282,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) tbinfo->attlen[j] = atoi(PQgetvalue(res, r, i_attlen)); tbinfo->attalign[j] = *(PQgetvalue(res, r, i_attalign)); tbinfo->attislocal[j] = (PQgetvalue(res, r, i_attislocal)[0] == 't'); + tbinfo->notnull_parent_invalid[j] = false; /* it only change in flagInhAttrs */ + tbinfo->notnull_invalid[j] = false; /* it only change in determineNotNullFlags */ /* Handle not-null constraint name and flags */ determineNotNullFlags(fout, res, r, @@ -9758,6 +9762,7 @@ determineNotNullFlags(Archive *fout, PGresult *res, int r, /* nothing else to do */ tbinfo->notnull_constrs[j] = NULL; + tbinfo->notnull_invalid[j] = true; return; } @@ -16986,6 +16991,18 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo) tbinfo->attrdefs[j]->adef_expr); } + /* + * if parent have invalid not-null, child have valid + * not-null, then we print not null on child too. later + * parent's invalid not-null will generate a ALTER TABLE ADD + * CONSTRAINT, which will cascade to children, which is + * fine. + */ + if (!print_notnull && + tbinfo->notnull_constrs[j] != NULL && + tbinfo->notnull_parent_invalid[j]) + print_notnull = true; + if (print_notnull) { if (tbinfo->notnull_constrs[j][0] == '\0') diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h index b426b5e4736..6c2f963dcc4 100644 --- a/src/bin/pg_dump/pg_dump.h +++ b/src/bin/pg_dump/pg_dump.h @@ -365,6 +365,11 @@ typedef struct _tableInfo * there isn't one on this column. If * empty string, unnamed constraint * (pre-v17) */ + bool *notnull_invalid; /* true for NOT NULL NOT VALID */ + + /* true if parent table NOT NULL constraint is NOT VALID */ + bool *notnull_parent_invalid; + bool *notnull_noinh; /* NOT NULL is NO INHERIT */ bool *notnull_islocal; /* true if NOT NULL has local definition */ struct _attrDefInfo **attrdefs; /* DEFAULT expressions */ -- 2.34.1 ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-23 16:18 Nathan Bossart <[email protected]> parent: jian he <[email protected]> 1 sibling, 1 reply; 11+ messages in thread From: Nathan Bossart @ 2025-04-23 16:18 UTC (permalink / raw) To: jian he <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers This one was briefly discussed in an RMT meeting. On Wed, Apr 09, 2025 at 01:16:20PM +0800, jian he wrote: > attached patch is for address pg_dump inconsistency > when parent is "not null not valid" while child is "not null". I see an open item [0] that points to this patch, but there's no owner listed, and there doesn't appear to have been any follow-up discussion. Álvaro, should I list you as the owner, and if so, are you planning to look at it soon? [0] https://wiki.postgresql.org/wiki/PostgreSQL_18_Open_Items -- nathan ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-24 19:36 Alvaro Herrera <[email protected]> parent: jian he <[email protected]> 1 sibling, 1 reply; 11+ messages in thread From: Alvaro Herrera @ 2025-04-24 19:36 UTC (permalink / raw) To: jian he <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; pgsql-hackers On 2025-Apr-09, jian he wrote: > hi. > > attached patch is for address pg_dump inconsistency > when parent is "not null not valid" while child is "not null". Here's my take on this patch. We don't really need the notnull_parent_invalid flag; in flagInhAttrs we can just set "islocal" to convince getTableAttrs to print the constraint. This also fixes the bug that in getTableAttrs() you handled the case where shouldPrintColumn() is true and not the other one. I also added test cases to pg_dump/t/002_pg_dump.pl to verify that the output was correct in those cases. In constraints.sql I added a couple of tables to ensure that the pg_upgrade handling (the pg_dump --binary-upgrade mode) is tested as well. Looking at the surrounding code in flagInhAttrs I noticed that we're mishandling this case: create table parent1 (a int); create table parent2 (a int); create table child () inherits (parent1, parent2); alter table parent1 add not null a; alter table parent2 add not null a not valid; We print the constraint for table child for no apparent reason. Patch 0002 is a part of your proposed patch that I don't think we need, but I'm open to hearing arguments about why we do, preferrably with some test cases. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "Hay que recordar que la existencia en el cosmos, y particularmente la elaboración de civilizaciones dentro de él no son, por desgracia, nada idílicas" (Ijon Tichy) Attachments: [text/x-diff] 0001-Fix-pg_dump-for-inherited-validated-not-null-constra.patch (9.5K, ../../[email protected]/2-0001-Fix-pg_dump-for-inherited-validated-not-null-constra.patch) download | inline diff: From 00c46ba90a24a99857cdc849943cf3fd7b96bd8e Mon Sep 17 00:00:00 2001 From: jian he <[email protected]> Date: Wed, 9 Apr 2025 13:07:58 +0800 Subject: [PATCH 1/2] Fix pg_dump for inherited validated not-null constraints When a child constraint is validated and its parent constraint isn't, pg_dump requires special handling. --- src/bin/pg_dump/common.c | 12 +++++++ src/bin/pg_dump/pg_dump.c | 12 ++++++- src/bin/pg_dump/pg_dump.h | 1 + src/bin/pg_dump/t/002_pg_dump.pl | 41 +++++++++++++++++++++-- src/test/regress/expected/constraints.out | 24 +++++++++++++ src/test/regress/sql/constraints.sql | 14 ++++++++ 6 files changed, 101 insertions(+), 3 deletions(-) diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c index 56b6c368acf..b0973d44f32 100644 --- a/src/bin/pg_dump/common.c +++ b/src/bin/pg_dump/common.c @@ -546,6 +546,18 @@ flagInhAttrs(Archive *fout, DumpOptions *dopt, TableInfo *tblinfo, int numTables parent->notnull_constrs[inhAttrInd] != NULL) foundNotNull = true; + /* + * For validated not-null constraints in child tables which + * derive from a parent constraint marked NOT VALID, we + * artificially mark the child constraint as local so that + * it is printed independently. Failing to do this would + * result in the child constraint being restored as NOT + * VALID. + */ + if (fout->remoteVersion >= 180000 && + parent->notnull_invalid[inhAttrInd]) + tbinfo->notnull_islocal[j] = true; + foundDefault |= (parentDef != NULL && strcmp(parentDef->adef_expr, "NULL") != 0 && !parent->attgenerated[inhAttrInd]); diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index 105e917aa7b..0e2485479f8 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -9255,6 +9255,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) tbinfo->attfdwoptions = (char **) pg_malloc(numatts * sizeof(char *)); tbinfo->attmissingval = (char **) pg_malloc(numatts * sizeof(char *)); tbinfo->notnull_constrs = (char **) pg_malloc(numatts * sizeof(char *)); + tbinfo->notnull_invalid = (bool *) pg_malloc(numatts * sizeof(bool)); tbinfo->notnull_noinh = (bool *) pg_malloc(numatts * sizeof(bool)); tbinfo->notnull_islocal = (bool *) pg_malloc(numatts * sizeof(bool)); tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(numatts * sizeof(AttrDefInfo *)); @@ -9280,6 +9281,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) tbinfo->attlen[j] = atoi(PQgetvalue(res, r, i_attlen)); tbinfo->attalign[j] = *(PQgetvalue(res, r, i_attalign)); tbinfo->attislocal[j] = (PQgetvalue(res, r, i_attislocal)[0] == 't'); + tbinfo->notnull_invalid[j] = false; /* it only change in + * determineNotNullFlags */ /* Handle not-null constraint name and flags */ determineNotNullFlags(fout, res, r, @@ -9756,6 +9759,12 @@ determineNotNullFlags(Archive *fout, PGresult *res, int r, else appendPQExpBuffer(*invalidnotnulloids, ",%s", constroid); + /* + * Track when a parent constraint is invalid for the cases where a + * child constraint has been validated independenly. + */ + tbinfo->notnull_invalid[j] = true; + /* nothing else to do */ tbinfo->notnull_constrs[j] = NULL; return; @@ -9763,10 +9772,11 @@ determineNotNullFlags(Archive *fout, PGresult *res, int r, /* * notnull_noinh is straight from the query result. notnull_islocal also, - * though flagInhAttrs may change that one later in versions < 18. + * though flagInhAttrs may change that one later. */ tbinfo->notnull_noinh[j] = PQgetvalue(res, r, i_notnull_noinherit)[0] == 't'; tbinfo->notnull_islocal[j] = PQgetvalue(res, r, i_notnull_islocal)[0] == 't'; + tbinfo->notnull_invalid[j] = false; /* * Determine a constraint name to use. If the column is not marked not- diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h index b426b5e4736..7417eab6aef 100644 --- a/src/bin/pg_dump/pg_dump.h +++ b/src/bin/pg_dump/pg_dump.h @@ -365,6 +365,7 @@ typedef struct _tableInfo * there isn't one on this column. If * empty string, unnamed constraint * (pre-v17) */ + bool *notnull_invalid; /* true for NOT NULL NOT VALID */ bool *notnull_noinh; /* NOT NULL is NO INHERIT */ bool *notnull_islocal; /* true if NOT NULL has local definition */ struct _attrDefInfo **attrdefs; /* DEFAULT expressions */ diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl index 6c03eca8e50..4494acaed8d 100644 --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -1118,10 +1118,17 @@ my %tests = ( }, }, - 'CONSTRAINT NOT NULL / INVALID' => { + 'CONSTRAINT NOT NULL / NOT VALID' => { create_sql => 'CREATE TABLE dump_test.test_table_nn ( col1 int); - ALTER TABLE dump_test.test_table_nn ADD CONSTRAINT nn NOT NULL col1 NOT VALID;', + CREATE TABLE dump_test.test_table_nn_chld1 ( + ) INHERITS (dump_test.test_table_nn); + CREATE TABLE dump_test.test_table_nn_chld2 ( + col1 int + ) INHERITS (dump_test.test_table_nn); + ALTER TABLE dump_test.test_table_nn ADD CONSTRAINT nn NOT NULL col1 NOT VALID; + ALTER TABLE dump_test.test_table_nn_chld1 VALIDATE CONSTRAINT nn; + ALTER TABLE dump_test.test_table_nn_chld2 VALIDATE CONSTRAINT nn;', regexp => qr/^ \QALTER TABLE dump_test.test_table_nn\E \n^\s+ \QADD CONSTRAINT nn NOT NULL col1 NOT VALID;\E @@ -1135,6 +1142,36 @@ my %tests = ( }, }, + 'CONSTRAINT NOT NULL / NOT VALID (child1)' => { + regexp => qr/^ + \QCREATE TABLE dump_test.test_table_nn_chld1 (\E\n + ^\s+\QCONSTRAINT nn NOT NULL col1\E$ + /xm, + like => { + %full_runs, %dump_test_schema_runs, section_pre_data => 1, + }, + unlike => { + exclude_dump_test_schema => 1, + only_dump_measurement => 1, + binary_upgrade => 1, + }, + }, + + 'CONSTRAINT NOT NULL / NOT VALID (child2)' => { + regexp => qr/^ + \QCREATE TABLE dump_test.test_table_nn_chld2 (\E\n + ^\s+\Qcol1 integer CONSTRAINT nn NOT NULL\E$ + /xm, + like => { + %full_runs, %dump_test_schema_runs, section_pre_data => 1, + }, + unlike => { + exclude_dump_test_schema => 1, + only_dump_measurement => 1, + }, + }, + + 'CONSTRAINT PRIMARY KEY / WITHOUT OVERLAPS' => { create_sql => 'CREATE TABLE dump_test.test_table_tpk ( col1 int4range, diff --git a/src/test/regress/expected/constraints.out b/src/test/regress/expected/constraints.out index 92e441a16cd..89f88cf9eb2 100644 --- a/src/test/regress/expected/constraints.out +++ b/src/test/regress/expected/constraints.out @@ -1625,6 +1625,30 @@ EXECUTE get_nnconstraint_info('{notnull_part1_upg, notnull_part1_1_upg, notnull_ notnull_part1_upg | notnull_con | f | t | 0 (4 rows) +-- Inheritance test tables for pg_upgrade +create table constr_parent (a int); +create table constr_child (a int) inherits (constr_parent); +NOTICE: merging column "a" with inherited definition +alter table constr_parent add not null a not valid; +alter table constr_child validate constraint constr_parent_a_not_null; +EXECUTE get_nnconstraint_info('{constr_parent, constr_child}'); + tabname | conname | convalidated | conislocal | coninhcount +---------------+--------------------------+--------------+------------+------------- + constr_child | constr_parent_a_not_null | t | f | 1 + constr_parent | constr_parent_a_not_null | f | t | 0 +(2 rows) + +create table constr_parent2 (a int); +create table constr_child2 () inherits (constr_parent2); +alter table constr_parent2 add not null a not valid; +alter table constr_child2 validate constraint constr_parent2_a_not_null; +EXECUTE get_nnconstraint_info('{constr_parent2, constr_child2}'); + tabname | conname | convalidated | conislocal | coninhcount +----------------+---------------------------+--------------+------------+------------- + constr_child2 | constr_parent2_a_not_null | t | f | 1 + constr_parent2 | constr_parent2_a_not_null | f | t | 0 +(2 rows) + DEALLOCATE get_nnconstraint_info; -- end NOT NULL NOT VALID -- Comments diff --git a/src/test/regress/sql/constraints.sql b/src/test/regress/sql/constraints.sql index 5d6d749c150..f93b4095044 100644 --- a/src/test/regress/sql/constraints.sql +++ b/src/test/regress/sql/constraints.sql @@ -979,6 +979,20 @@ INSERT INTO notnull_part1_3_upg values(NULL,1); ALTER TABLE notnull_part1_3_upg add CONSTRAINT nn3 NOT NULL a NOT VALID; ALTER TABLE notnull_part1_upg ATTACH PARTITION notnull_part1_3_upg FOR VALUES IN (NULL,5); EXECUTE get_nnconstraint_info('{notnull_part1_upg, notnull_part1_1_upg, notnull_part1_2_upg, notnull_part1_3_upg}'); + +-- Inheritance test tables for pg_upgrade +create table constr_parent (a int); +create table constr_child (a int) inherits (constr_parent); +alter table constr_parent add not null a not valid; +alter table constr_child validate constraint constr_parent_a_not_null; +EXECUTE get_nnconstraint_info('{constr_parent, constr_child}'); + +create table constr_parent2 (a int); +create table constr_child2 () inherits (constr_parent2); +alter table constr_parent2 add not null a not valid; +alter table constr_child2 validate constraint constr_parent2_a_not_null; +EXECUTE get_nnconstraint_info('{constr_parent2, constr_child2}'); + DEALLOCATE get_nnconstraint_info; -- end NOT NULL NOT VALID -- 2.39.5 [text/x-diff] 0002-Unnecessary-change.patch (1.1K, ../../[email protected]/3-0002-Unnecessary-change.patch) download | inline diff: From 91efc55ab422fcdff83fdc4fb9bd28b17c74c980 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <[email protected]> Date: Thu, 24 Apr 2025 21:09:22 +0200 Subject: [PATCH 2/2] Unnecessary change I think we don't have a good reason to do this. --- src/backend/catalog/pg_constraint.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index 70528679e57..6d5f83f9329 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -779,6 +779,17 @@ AdjustNotNullInheritance(Oid relid, AttrNumber attnum, ereport(ERROR, errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), errmsg("too many inheritance parents")); + + + /* + * If the child already has a valid constraint and we are creating + * an invalid one with same definition on it. The child's + * constraint will remain valid, but can no longer be marked as + * local. + */ + if (is_notvalid && conform->convalidated && conform->conenforced) + conform->conislocal = false; + changed = true; } else if (!conform->conislocal) -- 2.39.5 ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-25 09:02 Alvaro Herrera <[email protected]> parent: Nathan Bossart <[email protected]> 0 siblings, 0 replies; 11+ messages in thread From: Alvaro Herrera @ 2025-04-25 09:02 UTC (permalink / raw) To: Nathan Bossart <[email protected]>; +Cc: jian he <[email protected]>; Rushabh Lathia <[email protected]>; pgsql-hackers On 2025-Apr-23, Nathan Bossart wrote: > This one was briefly discussed in an RMT meeting. > > On Wed, Apr 09, 2025 at 01:16:20PM +0800, jian he wrote: > > attached patch is for address pg_dump inconsistency > > when parent is "not null not valid" while child is "not null". > > I see an open item [0] that points to this patch, but there's no owner > listed, and there doesn't appear to have been any follow-up discussion. > Álvaro, should I list you as the owner, and if so, are you planning to look > at it soon? Hello, yes, please list me as owner, and now at least there's a patch I feel more comfortable with :-) I'll leave it there for a couple of days for Jian and Rushabh to comment. Thanks, -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-26 04:00 jian he <[email protected]> parent: Alvaro Herrera <[email protected]> 0 siblings, 1 reply; 11+ messages in thread From: jian he @ 2025-04-26 04:00 UTC (permalink / raw) To: Alvaro Herrera <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; pgsql-hackers On Fri, Apr 25, 2025 at 3:36 AM Alvaro Herrera <[email protected]> wrote: > > On 2025-Apr-09, jian he wrote: > > > hi. > > > > attached patch is for address pg_dump inconsistency > > when parent is "not null not valid" while child is "not null". > > Here's my take on this patch. We don't really need the > notnull_parent_invalid flag; in flagInhAttrs we can just set "islocal" > to convince getTableAttrs to print the constraint. This also fixes the > bug that in getTableAttrs() you handled the case where > shouldPrintColumn() is true and not the other one. > Your patch is simpler than me. we indeed do not need the notnull_parent_invalid flag. I am wondering if we need to change the following comments in getTableAttrs. * We track in notnull_islocal whether the constraint was defined directly * in this table or via an ancestor, for binary upgrade. flagInhAttrs * might modify this later for servers older than 18; it's also in charge * of determining the correct inhcount. since we also changed notnull_islocal for pg18. Also do we need to adjust the following comments in determineNotNullFlags? * In a child table that inherits from a parent already containing NOT NULL * constraints and the columns in the child don't have their own NOT NULL * declarations, we suppress printing constraints in the child: the * constraints are acquired at the point where the child is attached to the * parent. This is tracked in ->notnull_islocal (which is set in flagInhAttrs * for servers pre-18). > > Looking at the surrounding code in flagInhAttrs I noticed that we're > mishandling this case: > > create table parent1 (a int); > create table parent2 (a int); > create table child () inherits (parent1, parent2); > alter table parent1 add not null a; > alter table parent2 add not null a not valid; > > We print the constraint for table child for no apparent reason. > > Patch 0002 is a part of your proposed patch that I don't think we need, > but I'm open to hearing arguments about why we do, preferrably with some > test cases. > ------------ CREATE TABLE inhnn (a int); insert into inhnn values(NULL); ALTER TABLE inhnn ADD CONSTRAINT cc not null a NOT VALID; CREATE TABLE inhnn_cc(a INTEGER) INHERITS(inhnn); CREATE TABLE inhnn_cc_1(a INTEGER) INHERITS(inhnn_cc, inhnn); -------- For the above sql scripts, the following query QUERYA, before and after dump (--clean --table-and-children=*inhnn* ) the results are the same. select conrelid::regclass::text, conname, convalidated, coninhcount, conislocal, conparentid, contype from pg_constraint where conrelid::regclass::text = ANY('{inhnn, inhnn_cc, inhnn_cc_1}') order by 1,2; without patch 0002: table before_dump; conrelid | conname | convalidated | coninhcount | conislocal | conparentid | contype ------------+---------+--------------+-------------+------------+-------------+--------- inhnn | cc | f | 0 | t | 0 | n inhnn_cc | cc | t | 1 | f | 0 | n inhnn_cc_1 | cc | t | 2 | f | 0 | n table after_dump; conrelid | conname | convalidated | coninhcount | conislocal | conparentid | contype ------------+---------+--------------+-------------+------------+-------------+--------- inhnn | cc | f | 0 | t | 0 | n inhnn_cc | cc | t | 1 | t | 0 | n inhnn_cc_1 | cc | t | 2 | t | 0 | n pg_dump --no-statistics --clean --table-and-children=*inhnn* --no-owner --verbose --column-inserts --file=dump.sql --no-acl in psql execute file "dump.sql", table after_dump is QUERYA's output using CTAS. As you can see, "conislocal" is not consistent, maybe in practical it does not matter, but here we can make pg_dump, "conislocal" value being consistent. ^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints @ 2025-04-28 14:50 Alvaro Herrera <[email protected]> parent: jian he <[email protected]> 0 siblings, 0 replies; 11+ messages in thread From: Alvaro Herrera @ 2025-04-28 14:50 UTC (permalink / raw) To: jian he <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; pgsql-hackers On 2025-Apr-26, jian he wrote: > I am wondering if we need to change the following comments in getTableAttrs. > > * We track in notnull_islocal whether the constraint was defined directly > * in this table or via an ancestor, for binary upgrade. flagInhAttrs > * might modify this later for servers older than 18; it's also in charge > * of determining the correct inhcount. > since we also changed notnull_islocal for pg18. Yeah. > Also do we need to adjust the following comments in determineNotNullFlags? > * In a child table that inherits from a parent already containing NOT NULL > * constraints and the columns in the child don't have their own NOT NULL > * declarations, we suppress printing constraints in the child: the > * constraints are acquired at the point where the child is attached to the > * parent. This is tracked in ->notnull_islocal (which is set in flagInhAttrs > * for servers pre-18). Adjusted this one as well. I also fixed the business with multiple inheritance: with the commit I just pushed, we stop printing the child constraint if at least one parent has a validated constraint. With that, I believe this open item is closed, so I'm going to mark it as such in the wiki page momentarily. > > Patch 0002 is a part of your proposed patch that I don't think we need, > > but I'm open to hearing arguments about why we do, preferrably with some > > test cases. > > ------------ > CREATE TABLE inhnn (a int); > insert into inhnn values(NULL); > ALTER TABLE inhnn ADD CONSTRAINT cc not null a NOT VALID; > CREATE TABLE inhnn_cc(a INTEGER) INHERITS(inhnn); > CREATE TABLE inhnn_cc_1(a INTEGER) INHERITS(inhnn_cc, inhnn); > -------- > As you can see, "conislocal" is not consistent, maybe in practical it > does not matter, > but here we can make pg_dump, "conislocal" value being consistent. Yeah, I realize that this effect exists, but I find it very hard to justify spending time on this issue given that there are no visible consequences. What I can offer is that if you come up with a test setup that fails when this patch is not applied, and works when it is applied, then I'm open to considering it. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ Syntax error: function hell() needs an argument. Please choose what hell you want to involve. ^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2025-04-28 14:50 UTC | newest] Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-12-16 20:24 [PATCH 6/8] BRIN bloom indexes Tomas Vondra <[email protected]> 2025-04-07 12:43 Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints jian he <[email protected]> 2025-04-07 17:24 ` Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints Alvaro Herrera <[email protected]> 2025-04-07 18:30 ` Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints Tom Lane <[email protected]> 2025-04-08 00:21 ` Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints Rushabh Lathia <[email protected]> 2025-04-09 05:16 ` Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints jian he <[email protected]> 2025-04-23 16:18 ` Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints Nathan Bossart <[email protected]> 2025-04-25 09:02 ` Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints Alvaro Herrera <[email protected]> 2025-04-24 19:36 ` Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints Alvaro Herrera <[email protected]> 2025-04-26 04:00 ` Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints jian he <[email protected]> 2025-04-28 14:50 ` Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints Alvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox