public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 6/8] BRIN bloom indexes
17+ messages / 6 participants
[nested] [flat]
* [PATCH 6/8] BRIN bloom indexes
@ 2020-12-16 20:24 Tomas Vondra <[email protected]>
0 siblings, 0 replies; 17+ messages in thread
From: Tomas Vondra @ 2020-12-16 20:24 UTC (permalink / raw)
Adds a BRIN opclass using a Bloom filter to summarize the range. BRIN
indexes using the new opclasses only allow equality queries, but that
works for data like UUID, MAC addresses etc. for which range queries are
not very common (and the data look random, making BRIN minmax indexes
inefficient anyway).
BRIN bloom opclasses allow specifying the usual Bloom parameters, like
expected number of distinct values (in the BRIN range) and desired
false positive rate.
The opclasses store 32-bit hashes of the values, not the raw indexed
values. This assumes the hash functions for data types has low number
of collisions, good performance etc. Collisions are not a huge issue
though, because the number of values in a BRIN ranges is fairly small.
Author: Tomas Vondra <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Alexander Korotkov <[email protected]>
Reviewed-by: Sokolov Yura <[email protected]>
Reviewed-by: John Naylor <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/5d78b774-7e9c-c94e-12cf-fef51cc89b1a%402ndquadrant.com
---
doc/src/sgml/brin.sgml | 178 +++++
doc/src/sgml/ref/create_index.sgml | 31 +
src/backend/access/brin/Makefile | 1 +
src/backend/access/brin/brin_bloom.c | 787 ++++++++++++++++++++++
src/include/access/brin.h | 2 +
src/include/access/brin_internal.h | 4 +
src/include/catalog/pg_amop.dat | 116 ++++
src/include/catalog/pg_amproc.dat | 447 ++++++++++++
src/include/catalog/pg_opclass.dat | 72 ++
src/include/catalog/pg_opfamily.dat | 38 ++
src/include/catalog/pg_proc.dat | 34 +
src/include/catalog/pg_type.dat | 7 +-
src/test/regress/expected/brin_bloom.out | 428 ++++++++++++
src/test/regress/expected/opr_sanity.out | 3 +-
src/test/regress/expected/psql.out | 3 +-
src/test/regress/expected/type_sanity.out | 7 +-
src/test/regress/parallel_schedule | 5 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/brin_bloom.sql | 376 +++++++++++
19 files changed, 2534 insertions(+), 6 deletions(-)
create mode 100644 src/backend/access/brin/brin_bloom.c
create mode 100644 src/test/regress/expected/brin_bloom.out
create mode 100644 src/test/regress/sql/brin_bloom.sql
diff --git a/doc/src/sgml/brin.sgml b/doc/src/sgml/brin.sgml
index 4420794e5b..577dd18c49 100644
--- a/doc/src/sgml/brin.sgml
+++ b/doc/src/sgml/brin.sgml
@@ -128,6 +128,10 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
</row>
</thead>
<tbody>
+ <row>
+ <entry valign="middle"><literal>int8_bloom_ops</literal></entry>
+ <entry><literal>= (bit,bit)</literal></entry>
+ </row>
<row>
<entry valign="middle" morerows="4"><literal>bit_minmax_ops</literal></entry>
<entry><literal>= (bit,bit)</literal></entry>
@@ -154,6 +158,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>|&> (box,box)</literal></entry></row>
<row><entry><literal>|>> (box,box)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>bpchar_bloom_ops</literal></entry>
+ <entry><literal>= (character,character)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>bpchar_minmax_ops</literal></entry>
<entry><literal>= (character,character)</literal></entry>
@@ -163,6 +172,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (character,character)</literal></entry></row>
<row><entry><literal>>= (character,character)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>bytea_bloom_ops</literal></entry>
+ <entry><literal>= (bytea,bytea)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>bytea_minmax_ops</literal></entry>
<entry><literal>= (bytea,bytea)</literal></entry>
@@ -172,6 +186,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (bytea,bytea)</literal></entry></row>
<row><entry><literal>>= (bytea,bytea)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>char_bloom_ops</literal></entry>
+ <entry><literal>= ("char","char")</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>char_minmax_ops</literal></entry>
<entry><literal>= ("char","char")</literal></entry>
@@ -181,6 +200,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> ("char","char")</literal></entry></row>
<row><entry><literal>>= ("char","char")</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>date_bloom_ops</literal></entry>
+ <entry><literal>= (date,date)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>date_minmax_ops</literal></entry>
<entry><literal>= (date,date)</literal></entry>
@@ -190,6 +214,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (date,date)</literal></entry></row>
<row><entry><literal>>= (date,date)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>float4_bloom_ops</literal></entry>
+ <entry><literal>= (float4,float4)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>float4_minmax_ops</literal></entry>
<entry><literal>= (float4,float4)</literal></entry>
@@ -199,6 +228,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal><= (float4,float4)</literal></entry></row>
<row><entry><literal>>= (float4,float4)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>float8_bloom_ops</literal></entry>
+ <entry><literal>= (float8,float8)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>float8_minmax_ops</literal></entry>
<entry><literal>= (float8,float8)</literal></entry>
@@ -218,6 +252,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>= (inet,inet)</literal></entry></row>
<row><entry><literal>&& (inet,inet)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>inet_bloom_ops</literal></entry>
+ <entry><literal>= (inet,inet)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>inet_minmax_ops</literal></entry>
<entry><literal>= (inet,inet)</literal></entry>
@@ -227,6 +266,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (inet,inet)</literal></entry></row>
<row><entry><literal>>= (inet,inet)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>int2_bloom_ops</literal></entry>
+ <entry><literal>= (int2,int2)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>int2_minmax_ops</literal></entry>
<entry><literal>= (int2,int2)</literal></entry>
@@ -236,6 +280,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal><= (int2,int2)</literal></entry></row>
<row><entry><literal>>= (int2,int2)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>int4_bloom_ops</literal></entry>
+ <entry><literal>= (int4,int4)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>int4_minmax_ops</literal></entry>
<entry><literal>= (int4,int4)</literal></entry>
@@ -245,6 +294,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal><= (int4,int4)</literal></entry></row>
<row><entry><literal>>= (int4,int4)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>int8_bloom_ops</literal></entry>
+ <entry><literal>= (bigint,bigint)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>int8_minmax_ops</literal></entry>
<entry><literal>= (bigint,bigint)</literal></entry>
@@ -254,6 +308,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal><= (bigint,bigint)</literal></entry></row>
<row><entry><literal>>= (bigint,bigint)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>interval_bloom_ops</literal></entry>
+ <entry><literal>= (interval,interval)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>interval_minmax_ops</literal></entry>
<entry><literal>= (interval,interval)</literal></entry>
@@ -263,6 +322,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (interval,interval)</literal></entry></row>
<row><entry><literal>>= (interval,interval)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>macaddr_bloom_ops</literal></entry>
+ <entry><literal>= (macaddr,macaddr)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>macaddr_minmax_ops</literal></entry>
<entry><literal>= (macaddr,macaddr)</literal></entry>
@@ -272,6 +336,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (macaddr,macaddr)</literal></entry></row>
<row><entry><literal>>= (macaddr,macaddr)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>macaddr8_bloom_ops</literal></entry>
+ <entry><literal>= (macaddr8,macaddr8)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>macaddr8_minmax_ops</literal></entry>
<entry><literal>= (macaddr8,macaddr8)</literal></entry>
@@ -281,6 +350,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (macaddr8,macaddr8)</literal></entry></row>
<row><entry><literal>>= (macaddr8,macaddr8)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>name_bloom_ops</literal></entry>
+ <entry><literal>= (name,name)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>name_minmax_ops</literal></entry>
<entry><literal>= (name,name)</literal></entry>
@@ -290,6 +364,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (name,name)</literal></entry></row>
<row><entry><literal>>= (name,name)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>numeric_bloom_ops</literal></entry>
+ <entry><literal>= (numeric,numeric)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>numeric_minmax_ops</literal></entry>
<entry><literal>= (numeric,numeric)</literal></entry>
@@ -299,6 +378,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (numeric,numeric)</literal></entry></row>
<row><entry><literal>>= (numeric,numeric)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>oid_bloom_ops</literal></entry>
+ <entry><literal>= (oid,oid)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>oid_minmax_ops</literal></entry>
<entry><literal>= (oid,oid)</literal></entry>
@@ -308,6 +392,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal><= (oid,oid)</literal></entry></row>
<row><entry><literal>>= (oid,oid)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>pg_lsn_bloom_ops</literal></entry>
+ <entry><literal>= (pg_lsn,pg_lsn)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>pg_lsn_minmax_ops</literal></entry>
<entry><literal>= (pg_lsn,pg_lsn)</literal></entry>
@@ -335,6 +424,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>&> (anyrange,anyrange)</literal></entry></row>
<row><entry><literal>-|- (anyrange,anyrange)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>text_bloom_ops</literal></entry>
+ <entry><literal>= (text,text)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>text_minmax_ops</literal></entry>
<entry><literal>= (text,text)</literal></entry>
@@ -344,6 +438,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (text,text)</literal></entry></row>
<row><entry><literal>>= (text,text)</literal></entry></row>
+ <row>
+ <entry valign="middle" morerows="4"><literal>tid_bloom_ops</literal></entry>
+ <entry><literal>= (tid,tid)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>tid_minmax_ops</literal></entry>
<entry><literal>= (tid,tid)</literal></entry>
@@ -353,6 +452,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal><= (tid,tid)</literal></entry></row>
<row><entry><literal>>= (tid,tid)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>timestamp_bloom_ops</literal></entry>
+ <entry><literal>= (timestamp,timestamp)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>timestamp_minmax_ops</literal></entry>
<entry><literal>= (timestamp,timestamp)</literal></entry>
@@ -362,6 +466,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (timestamp,timestamp)</literal></entry></row>
<row><entry><literal>>= (timestamp,timestamp)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>timestamptz_bloom_ops</literal></entry>
+ <entry><literal>= (timestamptz,timestamptz)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>timestamptz_minmax_ops</literal></entry>
<entry><literal>= (timestamptz,timestamptz)</literal></entry>
@@ -371,6 +480,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (timestamptz,timestamptz)</literal></entry></row>
<row><entry><literal>>= (timestamptz,timestamptz)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>time_bloom_ops</literal></entry>
+ <entry><literal>= (time,time)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>time_minmax_ops</literal></entry>
<entry><literal>= (time,time)</literal></entry>
@@ -380,6 +494,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (time,time)</literal></entry></row>
<row><entry><literal>>= (time,time)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>timetz_bloom_ops</literal></entry>
+ <entry><literal>= (timetz,timetz)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>timetz_minmax_ops</literal></entry>
<entry><literal>= (timetz,timetz)</literal></entry>
@@ -389,6 +508,11 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was
<row><entry><literal>> (timetz,timetz)</literal></entry></row>
<row><entry><literal>>= (timetz,timetz)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>uuid_bloom_ops</literal></entry>
+ <entry><literal>= (uuid,uuid)</literal></entry>
+ </row>
+
<row>
<entry valign="middle" morerows="4"><literal>uuid_minmax_ops</literal></entry>
<entry><literal>= (uuid,uuid)</literal></entry>
@@ -779,6 +903,60 @@ typedef struct BrinOpcInfo
function can improve index performance.
</para>
+ <para>
+ To write an operator class for a data type that implements only an equality
+ operator and supports hashing, it is possible to use the bloom support procedures
+ alongside the corresponding operators, as shown in
+ <xref linkend="brin-extensibility-bloom-table"/>.
+ All operator class members (procedures and operators) are mandatory.
+ </para>
+
+ <table id="brin-extensibility-bloom-table">
+ <title>Procedure and Support Numbers for Bloom Operator Classes</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Operator class member</entry>
+ <entry>Object</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>Support Procedure 1</entry>
+ <entry>internal function <function>brin_bloom_opcinfo()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 2</entry>
+ <entry>internal function <function>brin_bloom_add_value()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 3</entry>
+ <entry>internal function <function>brin_bloom_consistent()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 4</entry>
+ <entry>internal function <function>brin_bloom_union()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 11</entry>
+ <entry>function to compute hash of an element</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 1</entry>
+ <entry>operator equal-to</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ Support procedure numbers 1-10 are reserved for the BRIN internal
+ functions, so the SQL level functions start with number 11. Support
+ function number 11 is the main function required to build the index.
+ It should accept one argument with the same data type as the operator class,
+ and return a hash of the value.
+ </para>
+
<para>
Both minmax and inclusion operator classes support cross-data-type
operators, though with these the dependencies become more complicated.
diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/create_index.sgml
index 965dcf472c..a45691873c 100644
--- a/doc/src/sgml/ref/create_index.sgml
+++ b/doc/src/sgml/ref/create_index.sgml
@@ -581,6 +581,37 @@ CREATE [ UNIQUE ] INDEX [ CONCURRENTLY ] [ [ IF NOT EXISTS ] <replaceable class=
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>n_distinct_per_range</literal></term>
+ <listitem>
+ <para>
+ Defines the estimated number of distinct non-null values in the block
+ range, used by <acronym>BRIN</acronym> bloom indexes for sizing of the
+ Bloom filter. It behaves similarly to <literal>n_distinct</literal> option
+ for <xref linkend="sql-altertable"/>. When set to a positive value,
+ each block range is assumed to contain this number of distinct non-null
+ values. When set to a negative value, which must be greater than or
+ equal to -1, the number of distinct non-null is assumed linear with
+ the maximum possible number of tuples in the block range (about 290
+ rows per block). The default value is <literal>-0.1</literal>, and
+ the minimum number of distinct non-null values is <literal>16</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>false_positive_rate</literal></term>
+ <listitem>
+ <para>
+ Defines the desired false positive rate used by <acronym>BRIN</acronym>
+ bloom indexes for sizing of the Bloom filter. The values must be be
+ greater than 0.0 and smaller than 1.0. The default value is 0.01,
+ which is 1% false positive rate.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect2>
diff --git a/src/backend/access/brin/Makefile b/src/backend/access/brin/Makefile
index 468e1e289a..6b56131215 100644
--- a/src/backend/access/brin/Makefile
+++ b/src/backend/access/brin/Makefile
@@ -14,6 +14,7 @@ include $(top_builddir)/src/Makefile.global
OBJS = \
brin.o \
+ brin_bloom.o \
brin_inclusion.o \
brin_minmax.o \
brin_pageops.o \
diff --git a/src/backend/access/brin/brin_bloom.c b/src/backend/access/brin/brin_bloom.c
new file mode 100644
index 0000000000..4494484b3b
--- /dev/null
+++ b/src/backend/access/brin/brin_bloom.c
@@ -0,0 +1,787 @@
+/*
+ * brin_bloom.c
+ * Implementation of Bloom opclass for BRIN
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * A BRIN opclass summarizing page range into a bloom filter.
+ *
+ * Bloom filters allow efficient testing whether a given page range contains
+ * a particular value. Therefore, if we summarize each page range into a
+ * bloom filter, we can easily and cheaply test whether it contains values
+ * we get later.
+ *
+ * The index only supports equality operators, similarly to hash indexes.
+ * BRIN bloom indexes are however much smaller, and support only bitmap
+ * scans.
+ *
+ * Note: Don't confuse this with bloom indexes, implemented in a contrib
+ * module. That extension implements an entirely new AM, building a bloom
+ * filter on multiple columns in a single row. This opclass works with an
+ * existing AM (BRIN) and builds bloom filter on a column.
+ *
+ *
+ * values vs. hashes
+ * -----------------
+ *
+ * The original column values are not used directly, but are first hashed
+ * using the regular type-specific hash function, producing a uint32 hash.
+ * And this hash value is then added to the summary - i.e. it's hashed
+ * again and added to the bloom filter.
+ *
+ * This allows the code to treat all data types (byval/byref/...) the same
+ * way, with only minimal space requirements, because we're working with
+ * hashes and not the original values. Everything is uint32.
+ *
+ * Of course, this assumes the built-in hash function is reasonably good,
+ * without too many collisions etc. But that does seem to be the case, at
+ * least based on past experience. After all, the same hash functions are
+ * used for hash indexes, hash partitioning and so on.
+ *
+ *
+ * sizing the bloom filter
+ * -----------------------
+ *
+ * Size of a bloom filter depends on the number of distinct values we will
+ * store in it, and the desired false positive rate. The higher the number
+ * of distinct values and/or the lower the false positive rate, the larger
+ * the bloom filter. On the other hand, we want to keep the index as small
+ * as possible - that's one of the basic advantages of BRIN indexes.
+ *
+ * Although the number of distinct elements (in a page range) depends on
+ * the data, we can consider it fixed. This simplifies the trade-off to
+ * just false positive rate vs. size.
+ *
+ * At the page range level, false positive rate is a probability the bloom
+ * filter matches a random value. For the whole index (with sufficiently
+ * many page ranges) it represents the fraction of the index ranges (and
+ * thus fraction of the table to be scanned) matching the random value.
+ *
+ * Furthermore, the size of the bloom filter is subject to implementation
+ * limits - it has to fit onto a single index page (8kB by default). As
+ * the bitmap is inherently random, compression can't reliably help here.
+ * To reduce the size of a filter (to fit to a page), we have to either
+ * accept higher false positive rate (undesirable), or reduce the number
+ * of distinct items to be stored in the filter. We can't alter the input
+ * data, of course, but we may make the BRIN page ranges smaller - instead
+ * of the default 128 pages (1MB) we may build index with 16-page ranges,
+ * or something like that. This does help even for random data sets, as
+ * the number of rows per heap page is limited (to ~290 with very narrow
+ * tables, likely ~20 in practice).
+ *
+ * Of course, good sizing decisions depend on having the necessary data,
+ * i.e. number of distinct values in a page range (of a given size) and
+ * table size (to estimate cost change due to change in false positive
+ * rate due to having larger index vs. scanning larger indexes). We may
+ * not have that data - for example when building an index on empty table
+ * it's not really possible. And for some data we only have estimates for
+ * the whole table and we can only estimate per-range values (ndistinct).
+ *
+ * Another challenge is that while the bloom filter is per-column, it's
+ * the whole index tuple that has to fit into a page. And for multi-column
+ * indexes that may include pieces we have no control over (not necessarily
+ * bloom filters, the other columns may use other BRIN opclasses). So it's
+ * not entirely clear how to distrubute the space between those columns.
+ *
+ * The current logic, implemented in brin_bloom_get_ndistinct, attempts to
+ * make some basic sizing decisions, based on the size of BRIN ranges, and
+ * the maximum number of rows per range.
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/brin/brin_bloom.c
+ */
+#include "postgres.h"
+
+#include "access/genam.h"
+#include "access/brin.h"
+#include "access/brin_internal.h"
+#include "access/brin_page.h"
+#include "access/brin_tuple.h"
+#include "access/hash.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/stratnum.h"
+#include "catalog/pg_type.h"
+#include "catalog/pg_amop.h"
+#include "utils/builtins.h"
+#include "utils/datum.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+
+#include <math.h>
+
+#define BloomEqualStrategyNumber 1
+
+/*
+ * Additional SQL level support functions
+ *
+ * Procedure numbers must not use values reserved for BRIN itself; see
+ * brin_internal.h.
+ */
+#define BLOOM_MAX_PROCNUMS 1 /* maximum support procs we need */
+#define PROCNUM_HASH 11 /* required */
+
+/*
+ * Subtract this from procnum to obtain index in BloomOpaque arrays
+ * (Must be equal to minimum of private procnums).
+ */
+#define PROCNUM_BASE 11
+
+/*
+ * Storage type for BRIN's reloptions.
+ */
+typedef struct BloomOptions
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ double nDistinctPerRange; /* number of distinct values per range */
+ double falsePositiveRate; /* false positive for bloom filter */
+} BloomOptions;
+
+/*
+ * The current min value (16) is somewhat arbitrary, but it's based
+ * on the fact that the filter header is ~20B alone, which is about
+ * the same as the filter bitmap for 16 distinct items with 1% false
+ * positive rate. So by allowing lower values we'd not gain much. In
+ * any case, the min should not be larger than MaxHeapTuplesPerPage
+ * (~290), which is the theoretical maximum for single-page ranges.
+ */
+#define BLOOM_MIN_NDISTINCT_PER_RANGE 16
+
+/*
+ * Used to determine number of distinct items, based on the number of rows
+ * in a page range. The 10% is somewhat similar to what estimate_num_groups
+ * does, so we use the same factor here.
+ */
+#define BLOOM_DEFAULT_NDISTINCT_PER_RANGE -0.1 /* 10% of values */
+
+/*
+ * Allowed range and default value for the false positive range. The exact
+ * values are somewhat arbitrary, but were chosen considering the various
+ * parameters (size of filter vs. page size, etc.).
+ *
+ * The lower the false-positive rate, the more accurate the filter is, but
+ * it also gets larger - at some point this eliminates the main advantage
+ * of BRIN indexes, which is the tiny size. At 0.01% the index is about
+ * 10% of the table (assuming 290 distinct values per 8kB page).
+ *
+ * On the other hand, as the false-positive rate increases, larger part of
+ * the table has to be scanned due to mismatches - at 25% we're probably
+ * close to sequential scan being cheaper.
+ */
+#define BLOOM_MIN_FALSE_POSITIVE_RATE 0.0001 /* 0.01% fp rate */
+#define BLOOM_MAX_FALSE_POSITIVE_RATE 0.25 /* 25% fp rate */
+#define BLOOM_DEFAULT_FALSE_POSITIVE_RATE 0.01 /* 1% fp rate */
+
+#define BloomGetNDistinctPerRange(opts) \
+ ((opts) && (((BloomOptions *) (opts))->nDistinctPerRange != 0) ? \
+ (((BloomOptions *) (opts))->nDistinctPerRange) : \
+ BLOOM_DEFAULT_NDISTINCT_PER_RANGE)
+
+#define BloomGetFalsePositiveRate(opts) \
+ ((opts) && (((BloomOptions *) (opts))->falsePositiveRate != 0.0) ? \
+ (((BloomOptions *) (opts))->falsePositiveRate) : \
+ BLOOM_DEFAULT_FALSE_POSITIVE_RATE)
+
+
+#define BloomMaxFilterSize \
+ MAXALIGN_DOWN(BLCKSZ - \
+ (MAXALIGN(SizeOfPageHeaderData + \
+ sizeof(ItemIdData)) + \
+ MAXALIGN(sizeof(BrinSpecialSpace)) + \
+ SizeOfBrinTuple))
+
+
+/*
+ * Bloom Filter
+ *
+ * Represents a bloom filter, built on hashes of the indexed values. That is,
+ * we compute a uint32 hash of the value, and then store this hash into the
+ * bloom filter (and compute additional hashes on it).
+ *
+ * To calculate the additional hashes (treating the uint32 hash as an input
+ * value), we use the approach with two hash functions from this paper:
+ *
+ * Less Hashing, Same Performance:Building a Better Bloom Filter
+ * Adam Kirsch, Michael Mitzenmacher†, Harvard School of Engineering and
+ * Applied Sciences, Cambridge, Massachusetts [DOI 10.1002/rsa.20208]
+ *
+ * The two hash functions are calculated using hard-coded seeds.
+ *
+ * XXX We could implement "sparse" bloom filters, keeping only the bytes
+ * that are not entirely 0. But while indexes don't support TOAST, the
+ * varlena can still be compressed. So this seems unnecessary.
+ *
+ * XXX We can also watch the number of bits set in the bloom filter, and
+ * then stop using it (and not store the bitmap, to save space) when the
+ * false positive rate gets too high. But even if the false positive rate
+ * exceeds the desired value, it still can eliminate some page ranges.
+ */
+typedef struct BloomFilter
+{
+ /* varlena header (do not touch directly!) */
+ int32 vl_len_;
+
+ /* space for various flags (unused for now) */
+ uint16 flags;
+
+ /* fields for the HASHED phase */
+ uint8 nhashes; /* number of hash functions */
+ uint32 nbits; /* number of bits in the bitmap (size) */
+ uint32 nbits_set; /* number of bits set to 1 */
+
+ /* data of the bloom filter */
+ char data[FLEXIBLE_ARRAY_MEMBER];
+
+} BloomFilter;
+
+
+/*
+ * bloom_init
+ * Initialize the Bloom Filter, allocate all the memory.
+ *
+ * The filter is initialized with optimal size for ndistinct expected
+ * values, and requested false positive rate. The filter is stored as
+ * varlena.
+ */
+static BloomFilter *
+bloom_init(int ndistinct, double false_positive_rate)
+{
+ Size len;
+ BloomFilter *filter;
+
+ int nbits; /* size of filter / number of bits */
+ int nbytes; /* size of filter / number of bytes */
+
+ double k; /* number of hash functions */
+
+ Assert(ndistinct > 0);
+ Assert((false_positive_rate > 0) && (false_positive_rate < 1.0));
+
+ /* sizing bloom filter: -(n * ln(p)) / (ln(2))^2 */
+ nbits = ceil(-(ndistinct * log(false_positive_rate)) / pow(log(2.0), 2));
+
+ /* round m to whole bytes */
+ nbytes = ((nbits + 7) / 8);
+ nbits = nbytes * 8;
+
+ /*
+ * Reject filters that are obviously too large to store on a page.
+ *
+ * Initially the bloom filter is just zeroes and so very compressible, but
+ * as we add values it gets more and more random, and so less and less
+ * compressible. So initially everything fits on the page, but we might
+ * get surprising failures later - we want to prevent that, so we reject
+ * bloom filter that are obviously too large.
+ *
+ * XXX It's not uncommon to oversize the bloom filter a bit, to defend
+ * against unexpected data anomalies (parts of table with more distinct
+ * values per range etc.). But we still need to make sure even the
+ * oversized filter fits on page, if such need arises.
+ *
+ * XXX This check is not perfect, because the index may have multiple
+ * filters that are small individually, but too large when combined.
+ */
+ if (nbytes > BloomMaxFilterSize)
+ elog(ERROR, "the bloom filter is too large (%d > %zu)", nbytes,
+ BloomMaxFilterSize);
+
+ /*
+ * round(log(2.0) * m / ndistinct), but assume round() may not be
+ * available on Windows
+ */
+ k = log(2.0) * nbits / ndistinct;
+ k = (k - floor(k) >= 0.5) ? ceil(k) : floor(k);
+
+ /*
+ * We allocate the whole filter. Most of it is going to be 0 bits, so the
+ * varlena is easy to compress.
+ */
+ len = offsetof(BloomFilter, data) + nbytes;
+
+ filter = (BloomFilter *) palloc0(len);
+
+ filter->flags = 0;
+ filter->nhashes = (int) k;
+ filter->nbits = nbits;
+
+ SET_VARSIZE(filter, len);
+
+ return filter;
+}
+
+
+/*
+ * bloom_add_value
+ * Add value to the bloom filter.
+ */
+static BloomFilter *
+bloom_add_value(BloomFilter * filter, uint32 value, bool *updated)
+{
+ int i;
+ uint64 h1,
+ h2;
+
+ /* compute the hashes, used for the bloom filter */
+ h1 = DatumGetUInt64(hash_uint32_extended(value, 0x71d924af)) % filter->nbits;
+ h2 = DatumGetUInt64(hash_uint32_extended(value, 0xba48b314)) % filter->nbits;
+
+ /* compute the requested number of hashes */
+ for (i = 0; i < filter->nhashes; i++)
+ {
+ /* h1 + h2 + f(i) */
+ uint32 h = (h1 + i * h2) % filter->nbits;
+ uint32 byte = (h / 8);
+ uint32 bit = (h % 8);
+
+ /* if the bit is not set, set it and remember we did that */
+ if (!(filter->data[byte] & (0x01 << bit)))
+ {
+ filter->data[byte] |= (0x01 << bit);
+ filter->nbits_set++;
+ if (updated)
+ *updated = true;
+ }
+ }
+
+ return filter;
+}
+
+
+/*
+ * bloom_contains_value
+ * Check if the bloom filter contains a particular value.
+ */
+static bool
+bloom_contains_value(BloomFilter * filter, uint32 value)
+{
+ int i;
+ uint64 h1,
+ h2;
+
+ /* calculate the two hashes */
+ h1 = DatumGetUInt64(hash_uint32_extended(value, 0x71d924af)) % filter->nbits;
+ h2 = DatumGetUInt64(hash_uint32_extended(value, 0xba48b314)) % filter->nbits;
+
+ /* compute the requested number of hashes */
+ for (i = 0; i < filter->nhashes; i++)
+ {
+ /* h1 + h2 + f(i) */
+ uint32 h = (h1 + i * h2) % filter->nbits;
+ uint32 byte = (h / 8);
+ uint32 bit = (h % 8);
+
+ /* if the bit is not set, the value is not there */
+ if (!(filter->data[byte] & (0x01 << bit)))
+ return false;
+ }
+
+ /* all hashes found in bloom filter */
+ return true;
+}
+
+typedef struct BloomOpaque
+{
+ /*
+ * XXX At this point we only need a single proc (to compute the hash), but
+ * let's keep the array just like inclusion and minman opclasses, for
+ * consistency. We may need additional procs in the future.
+ */
+ FmgrInfo extra_procinfos[BLOOM_MAX_PROCNUMS];
+ bool extra_proc_missing[BLOOM_MAX_PROCNUMS];
+} BloomOpaque;
+
+static FmgrInfo *bloom_get_procinfo(BrinDesc *bdesc, uint16 attno,
+ uint16 procnum);
+
+
+Datum
+brin_bloom_opcinfo(PG_FUNCTION_ARGS)
+{
+ BrinOpcInfo *result;
+
+ /*
+ * opaque->strategy_procinfos is initialized lazily; here it is set to
+ * all-uninitialized by palloc0 which sets fn_oid to InvalidOid.
+ *
+ * bloom indexes only store the filter as a single BYTEA column
+ */
+
+ result = palloc0(MAXALIGN(SizeofBrinOpcInfo(1)) +
+ sizeof(BloomOpaque));
+ result->oi_nstored = 1;
+ result->oi_regular_nulls = true;
+ result->oi_opaque = (BloomOpaque *)
+ MAXALIGN((char *) result + SizeofBrinOpcInfo(1));
+ result->oi_typcache[0] = lookup_type_cache(PG_BRIN_BLOOM_SUMMARYOID, 0);
+
+ PG_RETURN_POINTER(result);
+}
+
+/*
+ * brin_bloom_get_ndistinct
+ * Determine the ndistinct value used to size bloom filter.
+ *
+ * Adjust the ndistinct value based on the pagesPerRange value. First,
+ * if it's negative, it's assumed to be relative to maximum number of
+ * tuples in the range (assuming each page gets MaxHeapTuplesPerPage
+ * tuples, which is likely a significant over-estimate). We also clamp
+ * the value, not to over-size the bloom filter unnecessarily.
+ *
+ * XXX We can only do this when the pagesPerRange value was supplied.
+ * If it wasn't, it has to be a read-only access to the index, in which
+ * case we don't really care. But perhaps we should fall-back to the
+ * default pagesPerRange value?
+ *
+ * XXX We might also fetch info about ndistinct estimate for the column,
+ * and compute the expected number of distinct values in a range. But
+ * that may be tricky due to data being sorted in various ways, so it
+ * seems better to rely on the upper estimate.
+ *
+ * XXX We might also calculate a better estimate of rows per BRIN range,
+ * instead of using MaxHeapTuplesPerPage (which probably produces values
+ * much higher than reality).
+ */
+static int
+brin_bloom_get_ndistinct(BrinDesc *bdesc, BloomOptions *opts)
+{
+ double ndistinct;
+ double maxtuples;
+ BlockNumber pagesPerRange;
+
+ pagesPerRange = BrinGetPagesPerRange(bdesc->bd_index);
+ ndistinct = BloomGetNDistinctPerRange(opts);
+
+ Assert(BlockNumberIsValid(pagesPerRange));
+
+ maxtuples = MaxHeapTuplesPerPage * pagesPerRange;
+
+ /*
+ * Similarly to n_distinct, negative values are relative - in this case to
+ * maximum number of tuples in the page range (maxtuples).
+ */
+ if (ndistinct < 0)
+ ndistinct = (-ndistinct) * maxtuples;
+
+ /*
+ * Positive values are to be used directly, but we still apply a couple of
+ * safeties no to use unreasonably small bloom filters.
+ */
+ ndistinct = Max(ndistinct, BLOOM_MIN_NDISTINCT_PER_RANGE);
+
+ /*
+ * And don't use more than the maximum possible number of tuples, in the
+ * range, which would be entirely wasteful.
+ */
+ ndistinct = Min(ndistinct, maxtuples);
+
+ return (int) ndistinct;
+}
+
+/*
+ * Examine the given index tuple (which contains partial status of a certain
+ * page range) by comparing it to the given value that comes from another heap
+ * tuple. If the new value is outside the bloom filter specified by the
+ * existing tuple values, update the index tuple and return true. Otherwise,
+ * return false and do not modify in this case.
+ */
+Datum
+brin_bloom_add_value(PG_FUNCTION_ARGS)
+{
+ BrinDesc *bdesc = (BrinDesc *) PG_GETARG_POINTER(0);
+ BrinValues *column = (BrinValues *) PG_GETARG_POINTER(1);
+ Datum newval = PG_GETARG_DATUM(2);
+ bool isnull PG_USED_FOR_ASSERTS_ONLY = PG_GETARG_DATUM(3);
+ BloomOptions *opts = (BloomOptions *) PG_GET_OPCLASS_OPTIONS();
+ Oid colloid = PG_GET_COLLATION();
+ FmgrInfo *hashFn;
+ uint32 hashValue;
+ bool updated = false;
+ AttrNumber attno;
+ BloomFilter *filter;
+
+ Assert(!isnull);
+
+ attno = column->bv_attno;
+
+ /*
+ * If this is the first non-null value, we need to initialize the bloom
+ * filter. Otherwise just extract the existing bloom filter from
+ * BrinValues.
+ */
+ if (column->bv_allnulls)
+ {
+ filter = bloom_init(brin_bloom_get_ndistinct(bdesc, opts),
+ BloomGetFalsePositiveRate(opts));
+ column->bv_values[0] = PointerGetDatum(filter);
+ column->bv_allnulls = false;
+ updated = true;
+ }
+ else
+ filter = (BloomFilter *) PG_DETOAST_DATUM(column->bv_values[0]);
+
+ /*
+ * Compute the hash of the new value, using the supplied hash function,
+ * and then add the hash value to the bloom filter.
+ */
+ hashFn = bloom_get_procinfo(bdesc, attno, PROCNUM_HASH);
+
+ hashValue = DatumGetUInt32(FunctionCall1Coll(hashFn, colloid, newval));
+
+ filter = bloom_add_value(filter, hashValue, &updated);
+
+ column->bv_values[0] = PointerGetDatum(filter);
+
+ PG_RETURN_BOOL(updated);
+}
+
+/*
+ * Given an index tuple corresponding to a certain page range and a scan key,
+ * return whether the scan key is consistent with the index tuple's bloom
+ * filter. Return true if so, false otherwise.
+ */
+Datum
+brin_bloom_consistent(PG_FUNCTION_ARGS)
+{
+ BrinDesc *bdesc = (BrinDesc *) PG_GETARG_POINTER(0);
+ BrinValues *column = (BrinValues *) PG_GETARG_POINTER(1);
+ ScanKey *keys = (ScanKey *) PG_GETARG_POINTER(2);
+ int nkeys = PG_GETARG_INT32(3);
+ Oid colloid = PG_GET_COLLATION();
+ AttrNumber attno;
+ Datum value;
+ Datum matches;
+ FmgrInfo *finfo;
+ uint32 hashValue;
+ BloomFilter *filter;
+ int keyno;
+
+ filter = (BloomFilter *) PG_DETOAST_DATUM(column->bv_values[0]);
+
+ Assert(filter);
+
+ matches = true;
+
+ for (keyno = 0; keyno < nkeys; keyno++)
+ {
+ ScanKey key = keys[keyno];
+
+ /* NULL keys are handled and filtered-out in bringetbitmap */
+ Assert(!(key->sk_flags & SK_ISNULL));
+
+ attno = key->sk_attno;
+ value = key->sk_argument;
+
+ switch (key->sk_strategy)
+ {
+ case BloomEqualStrategyNumber:
+
+ /*
+ * In the equality case (WHERE col = someval), we want to
+ * return the current page range if the minimum value in the
+ * range <= scan key, and the maximum value >= scan key.
+ */
+ finfo = bloom_get_procinfo(bdesc, attno, PROCNUM_HASH);
+
+ hashValue = DatumGetUInt32(FunctionCall1Coll(finfo, colloid, value));
+ matches &= bloom_contains_value(filter, hashValue);
+
+ break;
+ default:
+ /* shouldn't happen */
+ elog(ERROR, "invalid strategy number %d", key->sk_strategy);
+ matches = 0;
+ break;
+ }
+
+ if (!matches)
+ break;
+ }
+
+ PG_RETURN_DATUM(matches);
+}
+
+/*
+ * Given two BrinValues, update the first of them as a union of the summary
+ * values contained in both. The second one is untouched.
+ *
+ * XXX We assume the bloom filters have the same parameters for now. In the
+ * future we should have 'can union' function, to decide if we can combine
+ * two particular bloom filters.
+ */
+Datum
+brin_bloom_union(PG_FUNCTION_ARGS)
+{
+ int i;
+ int nbytes;
+ BrinValues *col_a = (BrinValues *) PG_GETARG_POINTER(1);
+ BrinValues *col_b = (BrinValues *) PG_GETARG_POINTER(2);
+ BloomFilter *filter_a;
+ BloomFilter *filter_b;
+
+ Assert(col_a->bv_attno == col_b->bv_attno);
+ Assert(!col_a->bv_allnulls && !col_b->bv_allnulls);
+
+ filter_a = (BloomFilter *) PG_DETOAST_DATUM(col_a->bv_values[0]);
+ filter_b = (BloomFilter *) PG_DETOAST_DATUM(col_b->bv_values[0]);
+
+ /* make sure the filters use the same parameters */
+ Assert(filter_a && filter_b);
+ Assert(filter_a->nbits == filter_b->nbits);
+ Assert(filter_a->nhashes == filter_b->nhashes);
+ Assert((filter_a->nbits > 0) && (filter_a->nbits % 8 == 0));
+
+ nbytes = (filter_a->nbits) / 8;
+
+ /* simply OR the bitmaps */
+ for (i = 0; i < nbytes; i++)
+ filter_a->data[i] |= filter_b->data[i];
+
+ PG_RETURN_VOID();
+}
+
+/*
+ * Cache and return inclusion opclass support procedure
+ *
+ * Return the procedure corresponding to the given function support number
+ * or null if it does not exist.
+ */
+static FmgrInfo *
+bloom_get_procinfo(BrinDesc *bdesc, uint16 attno, uint16 procnum)
+{
+ BloomOpaque *opaque;
+ uint16 basenum = procnum - PROCNUM_BASE;
+
+ /*
+ * We cache these in the opaque struct, to avoid repetitive syscache
+ * lookups.
+ */
+ opaque = (BloomOpaque *) bdesc->bd_info[attno - 1]->oi_opaque;
+
+ /*
+ * If we already searched for this proc and didn't find it, don't bother
+ * searching again.
+ */
+ if (opaque->extra_proc_missing[basenum])
+ return NULL;
+
+ if (opaque->extra_procinfos[basenum].fn_oid == InvalidOid)
+ {
+ if (RegProcedureIsValid(index_getprocid(bdesc->bd_index, attno,
+ procnum)))
+ {
+ fmgr_info_copy(&opaque->extra_procinfos[basenum],
+ index_getprocinfo(bdesc->bd_index, attno, procnum),
+ bdesc->bd_context);
+ }
+ else
+ {
+ opaque->extra_proc_missing[basenum] = true;
+ return NULL;
+ }
+ }
+
+ return &opaque->extra_procinfos[basenum];
+}
+
+Datum
+brin_bloom_options(PG_FUNCTION_ARGS)
+{
+ local_relopts *relopts = (local_relopts *) PG_GETARG_POINTER(0);
+
+ init_local_reloptions(relopts, sizeof(BloomOptions));
+
+ add_local_real_reloption(relopts, "n_distinct_per_range",
+ "number of distinct items expected in a BRIN page range",
+ BLOOM_DEFAULT_NDISTINCT_PER_RANGE,
+ -1.0, INT_MAX, offsetof(BloomOptions, nDistinctPerRange));
+
+ add_local_real_reloption(relopts, "false_positive_rate",
+ "desired false-positive rate for the bloom filters",
+ BLOOM_DEFAULT_FALSE_POSITIVE_RATE,
+ BLOOM_MIN_FALSE_POSITIVE_RATE,
+ BLOOM_MAX_FALSE_POSITIVE_RATE,
+ offsetof(BloomOptions, falsePositiveRate));
+
+ PG_RETURN_VOID();
+}
+
+/*
+ * brin_bloom_summary_in
+ * - input routine for type brin_bloom_summary.
+ *
+ * brin_bloom_summary is only used internally to represent summaries
+ * in BRIN bloom indexes, so it has no operations of its own, and we
+ * disallow input too.
+ */
+Datum
+brin_bloom_summary_in(PG_FUNCTION_ARGS)
+{
+ /*
+ * brin_bloom_summary stores the data in binary form and parsing text
+ * input is not needed, so disallow this.
+ */
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot accept a value of type %s", "pg_brin_bloom_summary")));
+
+ PG_RETURN_VOID(); /* keep compiler quiet */
+}
+
+
+/*
+ * brin_bloom_summary_out
+ * - output routine for type brin_bloom_summary.
+ *
+ * BRIN bloom summaries are serialized into a bytea value, but we want
+ * to output something nicer humans can understand.
+ */
+Datum
+brin_bloom_summary_out(PG_FUNCTION_ARGS)
+{
+ BloomFilter *filter;
+ StringInfoData str;
+
+ /* detoast the data to get value with a full 4B header */
+ filter = (BloomFilter *) PG_DETOAST_DATUM(PG_GETARG_BYTEA_PP(0));
+
+ initStringInfo(&str);
+ appendStringInfoChar(&str, '{');
+
+ appendStringInfo(&str, "mode: hashed nhashes: %u nbits: %u nbits_set: %u",
+ filter->nhashes, filter->nbits, filter->nbits_set);
+
+ appendStringInfoChar(&str, '}');
+
+ PG_RETURN_CSTRING(str.data);
+}
+
+/*
+ * brin_bloom_summary_recv
+ * - binary input routine for type brin_bloom_summary.
+ */
+Datum
+brin_bloom_summary_recv(PG_FUNCTION_ARGS)
+{
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot accept a value of type %s", "pg_brin_bloom_summary")));
+
+ PG_RETURN_VOID(); /* keep compiler quiet */
+}
+
+/*
+ * brin_bloom_summary_send
+ * - binary output routine for type brin_bloom_summary.
+ *
+ * BRIN bloom summaries are serialized in a bytea value (although the
+ * type is named differently), so let's just send that.
+ */
+Datum
+brin_bloom_summary_send(PG_FUNCTION_ARGS)
+{
+ return byteasend(fcinfo);
+}
diff --git a/src/include/access/brin.h b/src/include/access/brin.h
index 4e2be13cd6..0e52d75457 100644
--- a/src/include/access/brin.h
+++ b/src/include/access/brin.h
@@ -22,6 +22,8 @@ typedef struct BrinOptions
int32 vl_len_; /* varlena header (do not touch directly!) */
BlockNumber pagesPerRange;
bool autosummarize;
+ double nDistinctPerRange; /* number of distinct values per range */
+ double falsePositiveRate; /* false positive for bloom filter */
} BrinOptions;
diff --git a/src/include/access/brin_internal.h b/src/include/access/brin_internal.h
index 79440ebe7b..8cc4e532e6 100644
--- a/src/include/access/brin_internal.h
+++ b/src/include/access/brin_internal.h
@@ -58,6 +58,10 @@ typedef struct BrinDesc
/* total number of Datum entries that are stored on-disk for all columns */
int bd_totalstored;
+ /* parameters for sizing bloom filter (BRIN bloom opclasses) */
+ double bd_nDistinctPerRange;
+ double bd_falsePositiveRange;
+
/* per-column info; bd_tupdesc->natts entries long */
BrinOpcInfo *bd_info[FLEXIBLE_ARRAY_MEMBER];
} BrinDesc;
diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat
index 0f7ff63669..04d678f96a 100644
--- a/src/include/catalog/pg_amop.dat
+++ b/src/include/catalog/pg_amop.dat
@@ -1814,6 +1814,11 @@
amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)',
amopmethod => 'brin' },
+# bloom bytea
+{ amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea',
+ amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)',
+ amopmethod => 'brin' },
+
# minmax "char"
{ amopfamily => 'brin/char_minmax_ops', amoplefttype => 'char',
amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)',
@@ -1831,6 +1836,11 @@
amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)',
amopmethod => 'brin' },
+# bloom "char"
+{ amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char',
+ amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)',
+ amopmethod => 'brin' },
+
# minmax name
{ amopfamily => 'brin/name_minmax_ops', amoplefttype => 'name',
amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)',
@@ -1848,6 +1858,11 @@
amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)',
amopmethod => 'brin' },
+# bloom name
+{ amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name',
+ amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)',
+ amopmethod => 'brin' },
+
# minmax integer
{ amopfamily => 'brin/integer_minmax_ops', amoplefttype => 'int8',
@@ -1994,6 +2009,20 @@
amoprighttype => 'int8', amopstrategy => '5', amopopr => '>(int4,int8)',
amopmethod => 'brin' },
+# bloom integer
+
+{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int8',
+ amoprighttype => 'int8', amopstrategy => '1', amopopr => '=(int8,int8)',
+ amopmethod => 'brin' },
+
+{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int2',
+ amoprighttype => 'int2', amopstrategy => '1', amopopr => '=(int2,int2)',
+ amopmethod => 'brin' },
+
+{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int4',
+ amoprighttype => 'int4', amopstrategy => '1', amopopr => '=(int4,int4)',
+ amopmethod => 'brin' },
+
# minmax text
{ amopfamily => 'brin/text_minmax_ops', amoplefttype => 'text',
amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)',
@@ -2011,6 +2040,11 @@
amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)',
amopmethod => 'brin' },
+# bloom text
+{ amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text',
+ amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)',
+ amopmethod => 'brin' },
+
# minmax oid
{ amopfamily => 'brin/oid_minmax_ops', amoplefttype => 'oid',
amoprighttype => 'oid', amopstrategy => '1', amopopr => '<(oid,oid)',
@@ -2028,6 +2062,11 @@
amoprighttype => 'oid', amopstrategy => '5', amopopr => '>(oid,oid)',
amopmethod => 'brin' },
+# bloom oid
+{ amopfamily => 'brin/oid_bloom_ops', amoplefttype => 'oid',
+ amoprighttype => 'oid', amopstrategy => '1', amopopr => '=(oid,oid)',
+ amopmethod => 'brin' },
+
# minmax tid
{ amopfamily => 'brin/tid_minmax_ops', amoplefttype => 'tid',
amoprighttype => 'tid', amopstrategy => '1', amopopr => '<(tid,tid)',
@@ -2045,6 +2084,11 @@
amoprighttype => 'tid', amopstrategy => '5', amopopr => '>(tid,tid)',
amopmethod => 'brin' },
+# tid oid
+{ amopfamily => 'brin/tid_bloom_ops', amoplefttype => 'tid',
+ amoprighttype => 'tid', amopstrategy => '1', amopopr => '=(tid,tid)',
+ amopmethod => 'brin' },
+
# minmax float (float4, float8)
{ amopfamily => 'brin/float_minmax_ops', amoplefttype => 'float4',
@@ -2111,6 +2155,14 @@
amoprighttype => 'float8', amopstrategy => '5', amopopr => '>(float8,float8)',
amopmethod => 'brin' },
+# bloom float
+{ amopfamily => 'brin/float_bloom_ops', amoplefttype => 'float4',
+ amoprighttype => 'float4', amopstrategy => '1', amopopr => '=(float4,float4)',
+ amopmethod => 'brin' },
+{ amopfamily => 'brin/float_bloom_ops', amoplefttype => 'float8',
+ amoprighttype => 'float8', amopstrategy => '1', amopopr => '=(float8,float8)',
+ amopmethod => 'brin' },
+
# minmax macaddr
{ amopfamily => 'brin/macaddr_minmax_ops', amoplefttype => 'macaddr',
amoprighttype => 'macaddr', amopstrategy => '1',
@@ -2128,6 +2180,11 @@
amoprighttype => 'macaddr', amopstrategy => '5',
amopopr => '>(macaddr,macaddr)', amopmethod => 'brin' },
+# bloom macaddr
+{ amopfamily => 'brin/macaddr_bloom_ops', amoplefttype => 'macaddr',
+ amoprighttype => 'macaddr', amopstrategy => '1',
+ amopopr => '=(macaddr,macaddr)', amopmethod => 'brin' },
+
# minmax macaddr8
{ amopfamily => 'brin/macaddr8_minmax_ops', amoplefttype => 'macaddr8',
amoprighttype => 'macaddr8', amopstrategy => '1',
@@ -2145,6 +2202,11 @@
amoprighttype => 'macaddr8', amopstrategy => '5',
amopopr => '>(macaddr8,macaddr8)', amopmethod => 'brin' },
+# bloom macaddr8
+{ amopfamily => 'brin/macaddr8_bloom_ops', amoplefttype => 'macaddr8',
+ amoprighttype => 'macaddr8', amopstrategy => '1',
+ amopopr => '=(macaddr8,macaddr8)', amopmethod => 'brin' },
+
# minmax inet
{ amopfamily => 'brin/network_minmax_ops', amoplefttype => 'inet',
amoprighttype => 'inet', amopstrategy => '1', amopopr => '<(inet,inet)',
@@ -2162,6 +2224,11 @@
amoprighttype => 'inet', amopstrategy => '5', amopopr => '>(inet,inet)',
amopmethod => 'brin' },
+# bloom inet
+{ amopfamily => 'brin/network_bloom_ops', amoplefttype => 'inet',
+ amoprighttype => 'inet', amopstrategy => '1', amopopr => '=(inet,inet)',
+ amopmethod => 'brin' },
+
# inclusion inet
{ amopfamily => 'brin/network_inclusion_ops', amoplefttype => 'inet',
amoprighttype => 'inet', amopstrategy => '3', amopopr => '&&(inet,inet)',
@@ -2199,6 +2266,11 @@
amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)',
amopmethod => 'brin' },
+# bloom character
+{ amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar',
+ amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)',
+ amopmethod => 'brin' },
+
# minmax time without time zone
{ amopfamily => 'brin/time_minmax_ops', amoplefttype => 'time',
amoprighttype => 'time', amopstrategy => '1', amopopr => '<(time,time)',
@@ -2216,6 +2288,11 @@
amoprighttype => 'time', amopstrategy => '5', amopopr => '>(time,time)',
amopmethod => 'brin' },
+# bloom time without time zone
+{ amopfamily => 'brin/time_bloom_ops', amoplefttype => 'time',
+ amoprighttype => 'time', amopstrategy => '1', amopopr => '=(time,time)',
+ amopmethod => 'brin' },
+
# minmax datetime (date, timestamp, timestamptz)
{ amopfamily => 'brin/datetime_minmax_ops', amoplefttype => 'timestamp',
@@ -2362,6 +2439,20 @@
amoprighttype => 'timestamptz', amopstrategy => '5',
amopopr => '>(timestamptz,timestamptz)', amopmethod => 'brin' },
+# bloom datetime (date, timestamp, timestamptz)
+
+{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamp',
+ amoprighttype => 'timestamp', amopstrategy => '1',
+ amopopr => '=(timestamp,timestamp)', amopmethod => 'brin' },
+
+{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'date',
+ amoprighttype => 'date', amopstrategy => '1', amopopr => '=(date,date)',
+ amopmethod => 'brin' },
+
+{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamptz',
+ amoprighttype => 'timestamptz', amopstrategy => '1',
+ amopopr => '=(timestamptz,timestamptz)', amopmethod => 'brin' },
+
# minmax interval
{ amopfamily => 'brin/interval_minmax_ops', amoplefttype => 'interval',
amoprighttype => 'interval', amopstrategy => '1',
@@ -2379,6 +2470,11 @@
amoprighttype => 'interval', amopstrategy => '5',
amopopr => '>(interval,interval)', amopmethod => 'brin' },
+# bloom interval
+{ amopfamily => 'brin/interval_bloom_ops', amoplefttype => 'interval',
+ amoprighttype => 'interval', amopstrategy => '1',
+ amopopr => '=(interval,interval)', amopmethod => 'brin' },
+
# minmax time with time zone
{ amopfamily => 'brin/timetz_minmax_ops', amoplefttype => 'timetz',
amoprighttype => 'timetz', amopstrategy => '1', amopopr => '<(timetz,timetz)',
@@ -2396,6 +2492,11 @@
amoprighttype => 'timetz', amopstrategy => '5', amopopr => '>(timetz,timetz)',
amopmethod => 'brin' },
+# bloom time with time zone
+{ amopfamily => 'brin/timetz_bloom_ops', amoplefttype => 'timetz',
+ amoprighttype => 'timetz', amopstrategy => '1', amopopr => '=(timetz,timetz)',
+ amopmethod => 'brin' },
+
# minmax bit
{ amopfamily => 'brin/bit_minmax_ops', amoplefttype => 'bit',
amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)',
@@ -2447,6 +2548,11 @@
amoprighttype => 'numeric', amopstrategy => '5',
amopopr => '>(numeric,numeric)', amopmethod => 'brin' },
+# bloom numeric
+{ amopfamily => 'brin/numeric_bloom_ops', amoplefttype => 'numeric',
+ amoprighttype => 'numeric', amopstrategy => '1',
+ amopopr => '=(numeric,numeric)', amopmethod => 'brin' },
+
# minmax uuid
{ amopfamily => 'brin/uuid_minmax_ops', amoplefttype => 'uuid',
amoprighttype => 'uuid', amopstrategy => '1', amopopr => '<(uuid,uuid)',
@@ -2464,6 +2570,11 @@
amoprighttype => 'uuid', amopstrategy => '5', amopopr => '>(uuid,uuid)',
amopmethod => 'brin' },
+# bloom uuid
+{ amopfamily => 'brin/uuid_bloom_ops', amoplefttype => 'uuid',
+ amoprighttype => 'uuid', amopstrategy => '1', amopopr => '=(uuid,uuid)',
+ amopmethod => 'brin' },
+
# inclusion range types
{ amopfamily => 'brin/range_inclusion_ops', amoplefttype => 'anyrange',
amoprighttype => 'anyrange', amopstrategy => '1',
@@ -2525,6 +2636,11 @@
amoprighttype => 'pg_lsn', amopstrategy => '5', amopopr => '>(pg_lsn,pg_lsn)',
amopmethod => 'brin' },
+# bloom pg_lsn
+{ amopfamily => 'brin/pg_lsn_bloom_ops', amoplefttype => 'pg_lsn',
+ amoprighttype => 'pg_lsn', amopstrategy => '1', amopopr => '=(pg_lsn,pg_lsn)',
+ amopmethod => 'brin' },
+
# inclusion box
{ amopfamily => 'brin/box_inclusion_ops', amoplefttype => 'box',
amoprighttype => 'box', amopstrategy => '1', amopopr => '<<(box,box)',
diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat
index 36b5235c80..6709c8dfea 100644
--- a/src/include/catalog/pg_amproc.dat
+++ b/src/include/catalog/pg_amproc.dat
@@ -805,6 +805,24 @@
{ amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea',
amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom bytea
+{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea',
+ amprocrighttype => 'bytea', amprocnum => '11', amproc => 'hashvarlena' },
+
# minmax "char"
{ amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char',
amprocrighttype => 'char', amprocnum => '1',
@@ -818,6 +836,24 @@
{ amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char',
amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom "char"
+{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char',
+ amprocrighttype => 'char', amprocnum => '11', amproc => 'hashchar' },
+
# minmax name
{ amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name',
amprocrighttype => 'name', amprocnum => '1',
@@ -831,6 +867,24 @@
{ amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name',
amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom name
+{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name',
+ amprocrighttype => 'name', amprocnum => '11', amproc => 'hashname' },
+
# minmax integer: int2, int4, int8
{ amprocfamily => 'brin/integer_minmax_ops', amproclefttype => 'int8',
amprocrighttype => 'int8', amprocnum => '1',
@@ -932,6 +986,58 @@
{ amprocfamily => 'brin/integer_minmax_ops', amproclefttype => 'int4',
amprocrighttype => 'int2', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom integer: int2, int4, int8
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8',
+ amprocrighttype => 'int8', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8',
+ amprocrighttype => 'int8', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8',
+ amprocrighttype => 'int8', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8',
+ amprocrighttype => 'int8', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8',
+ amprocrighttype => 'int8', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8',
+ amprocrighttype => 'int8', amprocnum => '11', amproc => 'hashint8' },
+
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2',
+ amprocrighttype => 'int2', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2',
+ amprocrighttype => 'int2', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2',
+ amprocrighttype => 'int2', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2',
+ amprocrighttype => 'int2', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2',
+ amprocrighttype => 'int2', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2',
+ amprocrighttype => 'int2', amprocnum => '11', amproc => 'hashint2' },
+
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4',
+ amprocrighttype => 'int4', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4',
+ amprocrighttype => 'int4', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4',
+ amprocrighttype => 'int4', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4',
+ amprocrighttype => 'int4', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4',
+ amprocrighttype => 'int4', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4',
+ amprocrighttype => 'int4', amprocnum => '11', amproc => 'hashint4' },
+
# minmax text
{ amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text',
amprocrighttype => 'text', amprocnum => '1',
@@ -945,6 +1051,24 @@
{ amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text',
amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom text
+{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text',
+ amprocrighttype => 'text', amprocnum => '11', amproc => 'hashtext' },
+
# minmax oid
{ amprocfamily => 'brin/oid_minmax_ops', amproclefttype => 'oid',
amprocrighttype => 'oid', amprocnum => '1', amproc => 'brin_minmax_opcinfo' },
@@ -957,6 +1081,23 @@
{ amprocfamily => 'brin/oid_minmax_ops', amproclefttype => 'oid',
amprocrighttype => 'oid', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom oid
+{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid',
+ amprocrighttype => 'oid', amprocnum => '1', amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid',
+ amprocrighttype => 'oid', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid',
+ amprocrighttype => 'oid', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid',
+ amprocrighttype => 'oid', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid',
+ amprocrighttype => 'oid', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid',
+ amprocrighttype => 'oid', amprocnum => '11', amproc => 'hashoid' },
+
# minmax tid
{ amprocfamily => 'brin/tid_minmax_ops', amproclefttype => 'tid',
amprocrighttype => 'tid', amprocnum => '1', amproc => 'brin_minmax_opcinfo' },
@@ -969,6 +1110,23 @@
{ amprocfamily => 'brin/tid_minmax_ops', amproclefttype => 'tid',
amprocrighttype => 'tid', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom tid
+{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid',
+ amprocrighttype => 'tid', amprocnum => '1', amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid',
+ amprocrighttype => 'tid', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid',
+ amprocrighttype => 'tid', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid',
+ amprocrighttype => 'tid', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid',
+ amprocrighttype => 'tid', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/tid_bloom_ops', amproclefttype => 'tid',
+ amprocrighttype => 'tid', amprocnum => '11', amproc => 'hashtid' },
+
# minmax float
{ amprocfamily => 'brin/float_minmax_ops', amproclefttype => 'float4',
amprocrighttype => 'float4', amprocnum => '1',
@@ -1019,6 +1177,45 @@
amprocrighttype => 'float4', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# bloom float
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4',
+ amprocrighttype => 'float4', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4',
+ amprocrighttype => 'float4', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4',
+ amprocrighttype => 'float4', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4',
+ amprocrighttype => 'float4', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4',
+ amprocrighttype => 'float4', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4',
+ amprocrighttype => 'float4', amprocnum => '11',
+ amproc => 'hashfloat4' },
+
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8',
+ amprocrighttype => 'float8', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8',
+ amprocrighttype => 'float8', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8',
+ amprocrighttype => 'float8', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8',
+ amprocrighttype => 'float8', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8',
+ amprocrighttype => 'float8', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8',
+ amprocrighttype => 'float8', amprocnum => '11',
+ amproc => 'hashfloat8' },
+
# minmax macaddr
{ amprocfamily => 'brin/macaddr_minmax_ops', amproclefttype => 'macaddr',
amprocrighttype => 'macaddr', amprocnum => '1',
@@ -1033,6 +1230,26 @@
amprocrighttype => 'macaddr', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# bloom macaddr
+{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr',
+ amprocrighttype => 'macaddr', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr',
+ amprocrighttype => 'macaddr', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr',
+ amprocrighttype => 'macaddr', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr',
+ amprocrighttype => 'macaddr', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr',
+ amprocrighttype => 'macaddr', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr',
+ amprocrighttype => 'macaddr', amprocnum => '11',
+ amproc => 'hashmacaddr' },
+
# minmax macaddr8
{ amprocfamily => 'brin/macaddr8_minmax_ops', amproclefttype => 'macaddr8',
amprocrighttype => 'macaddr8', amprocnum => '1',
@@ -1047,6 +1264,26 @@
amprocrighttype => 'macaddr8', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# bloom macaddr8
+{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8',
+ amprocrighttype => 'macaddr8', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8',
+ amprocrighttype => 'macaddr8', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8',
+ amprocrighttype => 'macaddr8', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8',
+ amprocrighttype => 'macaddr8', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8',
+ amprocrighttype => 'macaddr8', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8',
+ amprocrighttype => 'macaddr8', amprocnum => '11',
+ amproc => 'hashmacaddr8' },
+
# minmax inet
{ amprocfamily => 'brin/network_minmax_ops', amproclefttype => 'inet',
amprocrighttype => 'inet', amprocnum => '1',
@@ -1060,6 +1297,24 @@
{ amprocfamily => 'brin/network_minmax_ops', amproclefttype => 'inet',
amprocrighttype => 'inet', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom inet
+{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet',
+ amprocrighttype => 'inet', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet',
+ amprocrighttype => 'inet', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet',
+ amprocrighttype => 'inet', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet',
+ amprocrighttype => 'inet', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet',
+ amprocrighttype => 'inet', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet',
+ amprocrighttype => 'inet', amprocnum => '11', amproc => 'hashinet' },
+
# inclusion inet
{ amprocfamily => 'brin/network_inclusion_ops', amproclefttype => 'inet',
amprocrighttype => 'inet', amprocnum => '1',
@@ -1094,6 +1349,26 @@
amprocrighttype => 'bpchar', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# bloom character
+{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar',
+ amprocrighttype => 'bpchar', amprocnum => '11',
+ amproc => 'hashchar' },
+
# minmax time without time zone
{ amprocfamily => 'brin/time_minmax_ops', amproclefttype => 'time',
amprocrighttype => 'time', amprocnum => '1',
@@ -1107,6 +1382,24 @@
{ amprocfamily => 'brin/time_minmax_ops', amproclefttype => 'time',
amprocrighttype => 'time', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom time without time zone
+{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time',
+ amprocrighttype => 'time', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time',
+ amprocrighttype => 'time', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time',
+ amprocrighttype => 'time', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time',
+ amprocrighttype => 'time', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time',
+ amprocrighttype => 'time', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time',
+ amprocrighttype => 'time', amprocnum => '11', amproc => 'time_hash' },
+
# minmax datetime (date, timestamp, timestamptz)
{ amprocfamily => 'brin/datetime_minmax_ops', amproclefttype => 'timestamp',
amprocrighttype => 'timestamp', amprocnum => '1',
@@ -1214,6 +1507,62 @@
amprocrighttype => 'timestamptz', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# bloom datetime (date, timestamp, timestamptz)
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp',
+ amprocrighttype => 'timestamp', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp',
+ amprocrighttype => 'timestamp', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp',
+ amprocrighttype => 'timestamp', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp',
+ amprocrighttype => 'timestamp', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp',
+ amprocrighttype => 'timestamp', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp',
+ amprocrighttype => 'timestamp', amprocnum => '11',
+ amproc => 'timestamp_hash' },
+
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz',
+ amprocrighttype => 'timestamptz', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz',
+ amprocrighttype => 'timestamptz', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz',
+ amprocrighttype => 'timestamptz', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz',
+ amprocrighttype => 'timestamptz', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz',
+ amprocrighttype => 'timestamptz', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz',
+ amprocrighttype => 'timestamptz', amprocnum => '11',
+ amproc => 'timestamp_hash' },
+
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date',
+ amprocrighttype => 'date', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date',
+ amprocrighttype => 'date', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date',
+ amprocrighttype => 'date', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date',
+ amprocrighttype => 'date', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date',
+ amprocrighttype => 'date', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date',
+ amprocrighttype => 'date', amprocnum => '11', amproc => 'hashint4' },
+
# minmax interval
{ amprocfamily => 'brin/interval_minmax_ops', amproclefttype => 'interval',
amprocrighttype => 'interval', amprocnum => '1',
@@ -1228,6 +1577,26 @@
amprocrighttype => 'interval', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# bloom interval
+{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval',
+ amprocrighttype => 'interval', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval',
+ amprocrighttype => 'interval', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval',
+ amprocrighttype => 'interval', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval',
+ amprocrighttype => 'interval', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval',
+ amprocrighttype => 'interval', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval',
+ amprocrighttype => 'interval', amprocnum => '11',
+ amproc => 'interval_hash' },
+
# minmax time with time zone
{ amprocfamily => 'brin/timetz_minmax_ops', amproclefttype => 'timetz',
amprocrighttype => 'timetz', amprocnum => '1',
@@ -1242,6 +1611,26 @@
amprocrighttype => 'timetz', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# bloom time with time zone
+{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz',
+ amprocrighttype => 'timetz', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz',
+ amprocrighttype => 'timetz', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz',
+ amprocrighttype => 'timetz', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz',
+ amprocrighttype => 'timetz', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz',
+ amprocrighttype => 'timetz', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz',
+ amprocrighttype => 'timetz', amprocnum => '11',
+ amproc => 'timetz_hash' },
+
# minmax bit
{ amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit',
amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_opcinfo' },
@@ -1282,6 +1671,26 @@
amprocrighttype => 'numeric', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# bloom numeric
+{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric',
+ amprocrighttype => 'numeric', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric',
+ amprocrighttype => 'numeric', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric',
+ amprocrighttype => 'numeric', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric',
+ amprocrighttype => 'numeric', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric',
+ amprocrighttype => 'numeric', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric',
+ amprocrighttype => 'numeric', amprocnum => '11',
+ amproc => 'hash_numeric' },
+
# minmax uuid
{ amprocfamily => 'brin/uuid_minmax_ops', amproclefttype => 'uuid',
amprocrighttype => 'uuid', amprocnum => '1',
@@ -1295,6 +1704,24 @@
{ amprocfamily => 'brin/uuid_minmax_ops', amproclefttype => 'uuid',
amprocrighttype => 'uuid', amprocnum => '4', amproc => 'brin_minmax_union' },
+# bloom uuid
+{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid',
+ amprocrighttype => 'uuid', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid',
+ amprocrighttype => 'uuid', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid',
+ amprocrighttype => 'uuid', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid',
+ amprocrighttype => 'uuid', amprocnum => '4', amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid',
+ amprocrighttype => 'uuid', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid',
+ amprocrighttype => 'uuid', amprocnum => '11', amproc => 'uuid_hash' },
+
# inclusion range types
{ amprocfamily => 'brin/range_inclusion_ops', amproclefttype => 'anyrange',
amprocrighttype => 'anyrange', amprocnum => '1',
@@ -1332,6 +1759,26 @@
amprocrighttype => 'pg_lsn', amprocnum => '4',
amproc => 'brin_minmax_union' },
+# bloom pg_lsn
+{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn',
+ amprocrighttype => 'pg_lsn', amprocnum => '1',
+ amproc => 'brin_bloom_opcinfo' },
+{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn',
+ amprocrighttype => 'pg_lsn', amprocnum => '2',
+ amproc => 'brin_bloom_add_value' },
+{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn',
+ amprocrighttype => 'pg_lsn', amprocnum => '3',
+ amproc => 'brin_bloom_consistent' },
+{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn',
+ amprocrighttype => 'pg_lsn', amprocnum => '4',
+ amproc => 'brin_bloom_union' },
+{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn',
+ amprocrighttype => 'pg_lsn', amprocnum => '5',
+ amproc => 'brin_bloom_options' },
+{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn',
+ amprocrighttype => 'pg_lsn', amprocnum => '11',
+ amproc => 'pg_lsn_hash' },
+
# inclusion box
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '1',
diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat
index 24b1433e1f..6a5bb58baf 100644
--- a/src/include/catalog/pg_opclass.dat
+++ b/src/include/catalog/pg_opclass.dat
@@ -266,67 +266,130 @@
{ opcmethod => 'brin', opcname => 'bytea_minmax_ops',
opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea',
opckeytype => 'bytea' },
+{ opcmethod => 'brin', opcname => 'bytea_bloom_ops',
+ opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea',
+ opckeytype => 'bytea', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'char_minmax_ops',
opcfamily => 'brin/char_minmax_ops', opcintype => 'char',
opckeytype => 'char' },
+{ opcmethod => 'brin', opcname => 'char_bloom_ops',
+ opcfamily => 'brin/char_bloom_ops', opcintype => 'char',
+ opckeytype => 'char', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'name_minmax_ops',
opcfamily => 'brin/name_minmax_ops', opcintype => 'name',
opckeytype => 'name' },
+{ opcmethod => 'brin', opcname => 'name_bloom_ops',
+ opcfamily => 'brin/name_bloom_ops', opcintype => 'name',
+ opckeytype => 'name', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'int8_minmax_ops',
opcfamily => 'brin/integer_minmax_ops', opcintype => 'int8',
opckeytype => 'int8' },
+{ opcmethod => 'brin', opcname => 'int8_bloom_ops',
+ opcfamily => 'brin/integer_bloom_ops', opcintype => 'int8',
+ opckeytype => 'int8', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'int2_minmax_ops',
opcfamily => 'brin/integer_minmax_ops', opcintype => 'int2',
opckeytype => 'int2' },
+{ opcmethod => 'brin', opcname => 'int2_bloom_ops',
+ opcfamily => 'brin/integer_bloom_ops', opcintype => 'int2',
+ opckeytype => 'int2', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'int4_minmax_ops',
opcfamily => 'brin/integer_minmax_ops', opcintype => 'int4',
opckeytype => 'int4' },
+{ opcmethod => 'brin', opcname => 'int4_bloom_ops',
+ opcfamily => 'brin/integer_bloom_ops', opcintype => 'int4',
+ opckeytype => 'int4', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'text_minmax_ops',
opcfamily => 'brin/text_minmax_ops', opcintype => 'text',
opckeytype => 'text' },
+{ opcmethod => 'brin', opcname => 'text_bloom_ops',
+ opcfamily => 'brin/text_bloom_ops', opcintype => 'text',
+ opckeytype => 'text', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'oid_minmax_ops',
opcfamily => 'brin/oid_minmax_ops', opcintype => 'oid', opckeytype => 'oid' },
+{ opcmethod => 'brin', opcname => 'oid_bloom_ops',
+ opcfamily => 'brin/oid_bloom_ops', opcintype => 'oid',
+ opckeytype => 'oid', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'tid_minmax_ops',
opcfamily => 'brin/tid_minmax_ops', opcintype => 'tid', opckeytype => 'tid' },
+{ opcmethod => 'brin', opcname => 'tid_bloom_ops',
+ opcfamily => 'brin/tid_bloom_ops', opcintype => 'tid', opckeytype => 'tid',
+ opcdefault => 'f'},
{ opcmethod => 'brin', opcname => 'float4_minmax_ops',
opcfamily => 'brin/float_minmax_ops', opcintype => 'float4',
opckeytype => 'float4' },
+{ opcmethod => 'brin', opcname => 'float4_bloom_ops',
+ opcfamily => 'brin/float_bloom_ops', opcintype => 'float4',
+ opckeytype => 'float4', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'float8_minmax_ops',
opcfamily => 'brin/float_minmax_ops', opcintype => 'float8',
opckeytype => 'float8' },
+{ opcmethod => 'brin', opcname => 'float8_bloom_ops',
+ opcfamily => 'brin/float_bloom_ops', opcintype => 'float8',
+ opckeytype => 'float8', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'macaddr_minmax_ops',
opcfamily => 'brin/macaddr_minmax_ops', opcintype => 'macaddr',
opckeytype => 'macaddr' },
+{ opcmethod => 'brin', opcname => 'macaddr_bloom_ops',
+ opcfamily => 'brin/macaddr_bloom_ops', opcintype => 'macaddr',
+ opckeytype => 'macaddr', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'macaddr8_minmax_ops',
opcfamily => 'brin/macaddr8_minmax_ops', opcintype => 'macaddr8',
opckeytype => 'macaddr8' },
+{ opcmethod => 'brin', opcname => 'macaddr8_bloom_ops',
+ opcfamily => 'brin/macaddr8_bloom_ops', opcintype => 'macaddr8',
+ opckeytype => 'macaddr8', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'inet_minmax_ops',
opcfamily => 'brin/network_minmax_ops', opcintype => 'inet',
opcdefault => 'f', opckeytype => 'inet' },
+{ opcmethod => 'brin', opcname => 'inet_bloom_ops',
+ opcfamily => 'brin/network_bloom_ops', opcintype => 'inet',
+ opcdefault => 'f', opckeytype => 'inet', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'inet_inclusion_ops',
opcfamily => 'brin/network_inclusion_ops', opcintype => 'inet',
opckeytype => 'inet' },
{ opcmethod => 'brin', opcname => 'bpchar_minmax_ops',
opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar',
opckeytype => 'bpchar' },
+{ opcmethod => 'brin', opcname => 'bpchar_bloom_ops',
+ opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar',
+ opckeytype => 'bpchar', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'time_minmax_ops',
opcfamily => 'brin/time_minmax_ops', opcintype => 'time',
opckeytype => 'time' },
+{ opcmethod => 'brin', opcname => 'time_bloom_ops',
+ opcfamily => 'brin/time_bloom_ops', opcintype => 'time',
+ opckeytype => 'time', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'date_minmax_ops',
opcfamily => 'brin/datetime_minmax_ops', opcintype => 'date',
opckeytype => 'date' },
+{ opcmethod => 'brin', opcname => 'date_bloom_ops',
+ opcfamily => 'brin/datetime_bloom_ops', opcintype => 'date',
+ opckeytype => 'date', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'timestamp_minmax_ops',
opcfamily => 'brin/datetime_minmax_ops', opcintype => 'timestamp',
opckeytype => 'timestamp' },
+{ opcmethod => 'brin', opcname => 'timestamp_bloom_ops',
+ opcfamily => 'brin/datetime_bloom_ops', opcintype => 'timestamp',
+ opckeytype => 'timestamp', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'timestamptz_minmax_ops',
opcfamily => 'brin/datetime_minmax_ops', opcintype => 'timestamptz',
opckeytype => 'timestamptz' },
+{ opcmethod => 'brin', opcname => 'timestamptz_bloom_ops',
+ opcfamily => 'brin/datetime_bloom_ops', opcintype => 'timestamptz',
+ opckeytype => 'timestamptz', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'interval_minmax_ops',
opcfamily => 'brin/interval_minmax_ops', opcintype => 'interval',
opckeytype => 'interval' },
+{ opcmethod => 'brin', opcname => 'interval_bloom_ops',
+ opcfamily => 'brin/interval_bloom_ops', opcintype => 'interval',
+ opckeytype => 'interval', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'timetz_minmax_ops',
opcfamily => 'brin/timetz_minmax_ops', opcintype => 'timetz',
opckeytype => 'timetz' },
+{ opcmethod => 'brin', opcname => 'timetz_bloom_ops',
+ opcfamily => 'brin/timetz_bloom_ops', opcintype => 'timetz',
+ opckeytype => 'timetz', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'bit_minmax_ops',
opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' },
{ opcmethod => 'brin', opcname => 'varbit_minmax_ops',
@@ -335,18 +398,27 @@
{ opcmethod => 'brin', opcname => 'numeric_minmax_ops',
opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric',
opckeytype => 'numeric' },
+{ opcmethod => 'brin', opcname => 'numeric_bloom_ops',
+ opcfamily => 'brin/numeric_bloom_ops', opcintype => 'numeric',
+ opckeytype => 'numeric', opcdefault => 'f' },
# no brin opclass for record, anyarray
{ opcmethod => 'brin', opcname => 'uuid_minmax_ops',
opcfamily => 'brin/uuid_minmax_ops', opcintype => 'uuid',
opckeytype => 'uuid' },
+{ opcmethod => 'brin', opcname => 'uuid_bloom_ops',
+ opcfamily => 'brin/uuid_bloom_ops', opcintype => 'uuid',
+ opckeytype => 'uuid', opcdefault => 'f' },
{ opcmethod => 'brin', opcname => 'range_inclusion_ops',
opcfamily => 'brin/range_inclusion_ops', opcintype => 'anyrange',
opckeytype => 'anyrange' },
{ opcmethod => 'brin', opcname => 'pg_lsn_minmax_ops',
opcfamily => 'brin/pg_lsn_minmax_ops', opcintype => 'pg_lsn',
opckeytype => 'pg_lsn' },
+{ opcmethod => 'brin', opcname => 'pg_lsn_bloom_ops',
+ opcfamily => 'brin/pg_lsn_bloom_ops', opcintype => 'pg_lsn',
+ opckeytype => 'pg_lsn', opcdefault => 'f' },
# no brin opclass for enum, tsvector, tsquery, jsonb
diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat
index 57e5aa0d8b..dea9adaf98 100644
--- a/src/include/catalog/pg_opfamily.dat
+++ b/src/include/catalog/pg_opfamily.dat
@@ -182,50 +182,88 @@
opfmethod => 'gin', opfname => 'jsonb_path_ops' },
{ oid => '4054',
opfmethod => 'brin', opfname => 'integer_minmax_ops' },
+{ oid => '9901',
+ opfmethod => 'brin', opfname => 'integer_bloom_ops' },
{ oid => '4055',
opfmethod => 'brin', opfname => 'numeric_minmax_ops' },
{ oid => '4056',
opfmethod => 'brin', opfname => 'text_minmax_ops' },
+{ oid => '9902',
+ opfmethod => 'brin', opfname => 'text_bloom_ops' },
+{ oid => '9903',
+ opfmethod => 'brin', opfname => 'numeric_bloom_ops' },
{ oid => '4058',
opfmethod => 'brin', opfname => 'timetz_minmax_ops' },
+{ oid => '9904',
+ opfmethod => 'brin', opfname => 'timetz_bloom_ops' },
{ oid => '4059',
opfmethod => 'brin', opfname => 'datetime_minmax_ops' },
+{ oid => '9905',
+ opfmethod => 'brin', opfname => 'datetime_bloom_ops' },
{ oid => '4062',
opfmethod => 'brin', opfname => 'char_minmax_ops' },
+{ oid => '9906',
+ opfmethod => 'brin', opfname => 'char_bloom_ops' },
{ oid => '4064',
opfmethod => 'brin', opfname => 'bytea_minmax_ops' },
+{ oid => '9907',
+ opfmethod => 'brin', opfname => 'bytea_bloom_ops' },
{ oid => '4065',
opfmethod => 'brin', opfname => 'name_minmax_ops' },
+{ oid => '9908',
+ opfmethod => 'brin', opfname => 'name_bloom_ops' },
{ oid => '4068',
opfmethod => 'brin', opfname => 'oid_minmax_ops' },
+{ oid => '9909',
+ opfmethod => 'brin', opfname => 'oid_bloom_ops' },
{ oid => '4069',
opfmethod => 'brin', opfname => 'tid_minmax_ops' },
+{ oid => '9910',
+ opfmethod => 'brin', opfname => 'tid_bloom_ops' },
{ oid => '4070',
opfmethod => 'brin', opfname => 'float_minmax_ops' },
+{ oid => '9911',
+ opfmethod => 'brin', opfname => 'float_bloom_ops' },
{ oid => '4074',
opfmethod => 'brin', opfname => 'macaddr_minmax_ops' },
+{ oid => '9912',
+ opfmethod => 'brin', opfname => 'macaddr_bloom_ops' },
{ oid => '4109',
opfmethod => 'brin', opfname => 'macaddr8_minmax_ops' },
+{ oid => '9913',
+ opfmethod => 'brin', opfname => 'macaddr8_bloom_ops' },
{ oid => '4075',
opfmethod => 'brin', opfname => 'network_minmax_ops' },
{ oid => '4102',
opfmethod => 'brin', opfname => 'network_inclusion_ops' },
+{ oid => '9914',
+ opfmethod => 'brin', opfname => 'network_bloom_ops' },
{ oid => '4076',
opfmethod => 'brin', opfname => 'bpchar_minmax_ops' },
+{ oid => '9915',
+ opfmethod => 'brin', opfname => 'bpchar_bloom_ops' },
{ oid => '4077',
opfmethod => 'brin', opfname => 'time_minmax_ops' },
+{ oid => '9916',
+ opfmethod => 'brin', opfname => 'time_bloom_ops' },
{ oid => '4078',
opfmethod => 'brin', opfname => 'interval_minmax_ops' },
+{ oid => '9917',
+ opfmethod => 'brin', opfname => 'interval_bloom_ops' },
{ oid => '4079',
opfmethod => 'brin', opfname => 'bit_minmax_ops' },
{ oid => '4080',
opfmethod => 'brin', opfname => 'varbit_minmax_ops' },
{ oid => '4081',
opfmethod => 'brin', opfname => 'uuid_minmax_ops' },
+{ oid => '9918',
+ opfmethod => 'brin', opfname => 'uuid_bloom_ops' },
{ oid => '4103',
opfmethod => 'brin', opfname => 'range_inclusion_ops' },
{ oid => '4082',
opfmethod => 'brin', opfname => 'pg_lsn_minmax_ops' },
+{ oid => '9919',
+ opfmethod => 'brin', opfname => 'pg_lsn_bloom_ops' },
{ oid => '4104',
opfmethod => 'brin', opfname => 'box_inclusion_ops' },
{ oid => '5000',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93bdb5ec83..bc38c2a144 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8217,6 +8217,26 @@
proargtypes => 'internal internal internal',
prosrc => 'brin_inclusion_union' },
+# BRIN bloom
+{ oid => '9920', descr => 'BRIN bloom support',
+ proname => 'brin_bloom_opcinfo', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'brin_bloom_opcinfo' },
+{ oid => '9921', descr => 'BRIN bloom support',
+ proname => 'brin_bloom_add_value', prorettype => 'bool',
+ proargtypes => 'internal internal internal internal',
+ prosrc => 'brin_bloom_add_value' },
+{ oid => '9922', descr => 'BRIN bloom support',
+ proname => 'brin_bloom_consistent', prorettype => 'bool',
+ proargtypes => 'internal internal internal int4',
+ prosrc => 'brin_bloom_consistent' },
+{ oid => '9923', descr => 'BRIN bloom support',
+ proname => 'brin_bloom_union', prorettype => 'bool',
+ proargtypes => 'internal internal internal',
+ prosrc => 'brin_bloom_union' },
+{ oid => '9924', descr => 'BRIN bloom support',
+ proname => 'brin_bloom_options', prorettype => 'void', proisstrict => 'f',
+ proargtypes => 'internal', prosrc => 'brin_bloom_options' },
+
# userlock replacements
{ oid => '2880', descr => 'obtain exclusive advisory lock',
proname => 'pg_advisory_lock', provolatile => 'v', proparallel => 'r',
@@ -11390,4 +11410,18 @@
proname => 'is_normalized', prorettype => 'bool', proargtypes => 'text text',
prosrc => 'unicode_is_normalized' },
+{ oid => '9035', descr => 'I/O',
+ proname => 'brin_bloom_summary_in', prorettype => 'pg_brin_bloom_summary',
+ proargtypes => 'cstring', prosrc => 'brin_bloom_summary_in' },
+{ oid => '9036', descr => 'I/O',
+ proname => 'brin_bloom_summary_out', prorettype => 'cstring',
+ proargtypes => 'pg_brin_bloom_summary', prosrc => 'brin_bloom_summary_out' },
+{ oid => '9037', descr => 'I/O',
+ proname => 'brin_bloom_summary_recv', provolatile => 's',
+ prorettype => 'pg_brin_bloom_summary', proargtypes => 'internal',
+ prosrc => 'brin_bloom_summary_recv' },
+{ oid => '9038', descr => 'I/O',
+ proname => 'brin_bloom_summary_send', provolatile => 's', prorettype => 'bytea',
+ proargtypes => 'pg_brin_bloom_summary', prosrc => 'brin_bloom_summary_send' },
+
]
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f53b..74e279cbf9 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -679,5 +679,10 @@
typtype => 'p', typcategory => 'P', typinput => 'anycompatiblemultirange_in',
typoutput => 'anycompatiblemultirange_out', typreceive => '-', typsend => '-',
typalign => 'd', typstorage => 'x' },
-
+{ oid => '9925',
+ descr => 'BRIN bloom summary',
+ typname => 'pg_brin_bloom_summary', typlen => '-1', typbyval => 'f', typcategory => 'S',
+ typinput => 'brin_bloom_summary_in', typoutput => 'brin_bloom_summary_out',
+ typreceive => 'brin_bloom_summary_recv', typsend => 'brin_bloom_summary_send',
+ typalign => 'i', typstorage => 'x', typcollation => 'default' },
]
diff --git a/src/test/regress/expected/brin_bloom.out b/src/test/regress/expected/brin_bloom.out
new file mode 100644
index 0000000000..32c56a996a
--- /dev/null
+++ b/src/test/regress/expected/brin_bloom.out
@@ -0,0 +1,428 @@
+CREATE TABLE brintest_bloom (byteacol bytea,
+ charcol "char",
+ namecol name,
+ int8col bigint,
+ int2col smallint,
+ int4col integer,
+ textcol text,
+ oidcol oid,
+ float4col real,
+ float8col double precision,
+ macaddrcol macaddr,
+ inetcol inet,
+ cidrcol cidr,
+ bpcharcol character,
+ datecol date,
+ timecol time without time zone,
+ timestampcol timestamp without time zone,
+ timestamptzcol timestamp with time zone,
+ intervalcol interval,
+ timetzcol time with time zone,
+ numericcol numeric,
+ uuidcol uuid,
+ lsncol pg_lsn
+) WITH (fillfactor=10);
+INSERT INTO brintest_bloom SELECT
+ repeat(stringu1, 8)::bytea,
+ substr(stringu1, 1, 1)::"char",
+ stringu1::name, 142857 * tenthous,
+ thousand,
+ twothousand,
+ repeat(stringu1, 8),
+ unique1::oid,
+ (four + 1.0)/(hundred+1),
+ odd::float8 / (tenthous + 1),
+ format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr,
+ inet '10.2.3.4/24' + tenthous,
+ cidr '10.2.3/24' + tenthous,
+ substr(stringu1, 1, 1)::bpchar,
+ date '1995-08-15' + tenthous,
+ time '01:20:30' + thousand * interval '18.5 second',
+ timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours',
+ timestamptz '1972-10-10 03:00' + thousand * interval '1 hour',
+ justify_days(justify_hours(tenthous * interval '12 minutes')),
+ timetz '01:30:20+02' + hundred * interval '15 seconds',
+ tenthous::numeric(36,30) * fivethous * even / (hundred + 1),
+ format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid,
+ format('%s/%s%s', odd, even, tenthous)::pg_lsn
+FROM tenk1 ORDER BY unique2 LIMIT 100;
+-- throw in some NULL's and different values
+INSERT INTO brintest_bloom (inetcol, cidrcol) SELECT
+ inet 'fe80::6e40:8ff:fea9:8c46' + tenthous,
+ cidr 'fe80::6e40:8ff:fea9:8c46' + tenthous
+FROM tenk1 ORDER BY thousand, tenthous LIMIT 25;
+-- test bloom specific index options
+-- ndistinct must be >= -1.0
+CREATE INDEX brinidx_bloom ON brintest_bloom USING brin (
+ byteacol bytea_bloom_ops(n_distinct_per_range = -1.1)
+);
+ERROR: value -1.1 out of bounds for option "n_distinct_per_range"
+DETAIL: Valid values are between "-1.000000" and "2147483647.000000".
+-- false_positive_rate must be between 0.0001 and 0.25
+CREATE INDEX brinidx_bloom ON brintest_bloom USING brin (
+ byteacol bytea_bloom_ops(false_positive_rate = 0.00009)
+);
+ERROR: value 0.00009 out of bounds for option "false_positive_rate"
+DETAIL: Valid values are between "0.000100" and "0.250000".
+CREATE INDEX brinidx_bloom ON brintest_bloom USING brin (
+ byteacol bytea_bloom_ops(false_positive_rate = 0.26)
+);
+ERROR: value 0.26 out of bounds for option "false_positive_rate"
+DETAIL: Valid values are between "0.000100" and "0.250000".
+CREATE INDEX brinidx_bloom ON brintest_bloom USING brin (
+ byteacol bytea_bloom_ops,
+ charcol char_bloom_ops,
+ namecol name_bloom_ops,
+ int8col int8_bloom_ops,
+ int2col int2_bloom_ops,
+ int4col int4_bloom_ops,
+ textcol text_bloom_ops,
+ oidcol oid_bloom_ops,
+ float4col float4_bloom_ops,
+ float8col float8_bloom_ops,
+ macaddrcol macaddr_bloom_ops,
+ inetcol inet_bloom_ops,
+ cidrcol inet_bloom_ops,
+ bpcharcol bpchar_bloom_ops,
+ datecol date_bloom_ops,
+ timecol time_bloom_ops,
+ timestampcol timestamp_bloom_ops,
+ timestamptzcol timestamptz_bloom_ops,
+ intervalcol interval_bloom_ops,
+ timetzcol timetz_bloom_ops,
+ numericcol numeric_bloom_ops,
+ uuidcol uuid_bloom_ops,
+ lsncol pg_lsn_bloom_ops
+) with (pages_per_range = 1);
+CREATE TABLE brinopers_bloom (colname name, typ text,
+ op text[], value text[], matches int[],
+ check (cardinality(op) = cardinality(value)),
+ check (cardinality(op) = cardinality(matches)));
+INSERT INTO brinopers_bloom VALUES
+ ('byteacol', 'bytea',
+ '{=}',
+ '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}',
+ '{1}'),
+ ('charcol', '"char"',
+ '{=}',
+ '{M}',
+ '{6}'),
+ ('namecol', 'name',
+ '{=}',
+ '{MAAAAA}',
+ '{2}'),
+ ('int2col', 'int2',
+ '{=}',
+ '{800}',
+ '{1}'),
+ ('int4col', 'int4',
+ '{=}',
+ '{800}',
+ '{1}'),
+ ('int8col', 'int8',
+ '{=}',
+ '{1257141600}',
+ '{1}'),
+ ('textcol', 'text',
+ '{=}',
+ '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}',
+ '{1}'),
+ ('oidcol', 'oid',
+ '{=}',
+ '{8800}',
+ '{1}'),
+ ('float4col', 'float4',
+ '{=}',
+ '{1}',
+ '{4}'),
+ ('float8col', 'float8',
+ '{=}',
+ '{0}',
+ '{1}'),
+ ('macaddrcol', 'macaddr',
+ '{=}',
+ '{2c:00:2d:00:16:00}',
+ '{2}'),
+ ('inetcol', 'inet',
+ '{=}',
+ '{10.2.14.231/24}',
+ '{1}'),
+ ('inetcol', 'cidr',
+ '{=}',
+ '{fe80::6e40:8ff:fea9:8c46}',
+ '{1}'),
+ ('cidrcol', 'inet',
+ '{=}',
+ '{10.2.14/24}',
+ '{2}'),
+ ('cidrcol', 'inet',
+ '{=}',
+ '{fe80::6e40:8ff:fea9:8c46}',
+ '{1}'),
+ ('cidrcol', 'cidr',
+ '{=}',
+ '{10.2.14/24}',
+ '{2}'),
+ ('cidrcol', 'cidr',
+ '{=}',
+ '{fe80::6e40:8ff:fea9:8c46}',
+ '{1}'),
+ ('bpcharcol', 'bpchar',
+ '{=}',
+ '{W}',
+ '{6}'),
+ ('datecol', 'date',
+ '{=}',
+ '{2009-12-01}',
+ '{1}'),
+ ('timecol', 'time',
+ '{=}',
+ '{02:28:57}',
+ '{1}'),
+ ('timestampcol', 'timestamp',
+ '{=}',
+ '{1964-03-24 19:26:45}',
+ '{1}'),
+ ('timestamptzcol', 'timestamptz',
+ '{=}',
+ '{1972-10-19 09:00:00-07}',
+ '{1}'),
+ ('intervalcol', 'interval',
+ '{=}',
+ '{1 mons 13 days 12:24}',
+ '{1}'),
+ ('timetzcol', 'timetz',
+ '{=}',
+ '{01:35:50+02}',
+ '{2}'),
+ ('numericcol', 'numeric',
+ '{=}',
+ '{2268164.347826086956521739130434782609}',
+ '{1}'),
+ ('uuidcol', 'uuid',
+ '{=}',
+ '{52225222-5222-5222-5222-522252225222}',
+ '{1}'),
+ ('lsncol', 'pg_lsn',
+ '{=, IS, IS NOT}',
+ '{44/455222, NULL, NULL}',
+ '{1, 25, 100}');
+DO $x$
+DECLARE
+ r record;
+ r2 record;
+ cond text;
+ idx_ctids tid[];
+ ss_ctids tid[];
+ count int;
+ plan_ok bool;
+ plan_line text;
+BEGIN
+ FOR r IN SELECT colname, oper, typ, value[ordinality], matches[ordinality] FROM brinopers_bloom, unnest(op) WITH ORDINALITY AS oper LOOP
+
+ -- prepare the condition
+ IF r.value IS NULL THEN
+ cond := format('%I %s %L', r.colname, r.oper, r.value);
+ ELSE
+ cond := format('%I %s %L::%s', r.colname, r.oper, r.value, r.typ);
+ END IF;
+
+ -- run the query using the brin index
+ SET enable_seqscan = 0;
+ SET enable_bitmapscan = 1;
+
+ plan_ok := false;
+ FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP
+ IF plan_line LIKE '%Bitmap Heap Scan on brintest_bloom%' THEN
+ plan_ok := true;
+ END IF;
+ END LOOP;
+ IF NOT plan_ok THEN
+ RAISE WARNING 'did not get bitmap indexscan plan for %', r;
+ END IF;
+
+ EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond)
+ INTO idx_ctids;
+
+ -- run the query using a seqscan
+ SET enable_seqscan = 1;
+ SET enable_bitmapscan = 0;
+
+ plan_ok := false;
+ FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP
+ IF plan_line LIKE '%Seq Scan on brintest_bloom%' THEN
+ plan_ok := true;
+ END IF;
+ END LOOP;
+ IF NOT plan_ok THEN
+ RAISE WARNING 'did not get seqscan plan for %', r;
+ END IF;
+
+ EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond)
+ INTO ss_ctids;
+
+ -- make sure both return the same results
+ count := array_length(idx_ctids, 1);
+
+ IF NOT (count = array_length(ss_ctids, 1) AND
+ idx_ctids @> ss_ctids AND
+ idx_ctids <@ ss_ctids) THEN
+ -- report the results of each scan to make the differences obvious
+ RAISE WARNING 'something not right in %: count %', r, count;
+ SET enable_seqscan = 1;
+ SET enable_bitmapscan = 0;
+ FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP
+ RAISE NOTICE 'seqscan: %', r2;
+ END LOOP;
+
+ SET enable_seqscan = 0;
+ SET enable_bitmapscan = 1;
+ FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP
+ RAISE NOTICE 'bitmapscan: %', r2;
+ END LOOP;
+ END IF;
+
+ -- make sure we found expected number of matches
+ IF count != r.matches THEN RAISE WARNING 'unexpected number of results % for %', count, r; END IF;
+ END LOOP;
+END;
+$x$;
+RESET enable_seqscan;
+RESET enable_bitmapscan;
+INSERT INTO brintest_bloom SELECT
+ repeat(stringu1, 42)::bytea,
+ substr(stringu1, 1, 1)::"char",
+ stringu1::name, 142857 * tenthous,
+ thousand,
+ twothousand,
+ repeat(stringu1, 42),
+ unique1::oid,
+ (four + 1.0)/(hundred+1),
+ odd::float8 / (tenthous + 1),
+ format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr,
+ inet '10.2.3.4' + tenthous,
+ cidr '10.2.3/24' + tenthous,
+ substr(stringu1, 1, 1)::bpchar,
+ date '1995-08-15' + tenthous,
+ time '01:20:30' + thousand * interval '18.5 second',
+ timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours',
+ timestamptz '1972-10-10 03:00' + thousand * interval '1 hour',
+ justify_days(justify_hours(tenthous * interval '12 minutes')),
+ timetz '01:30:20' + hundred * interval '15 seconds',
+ tenthous::numeric(36,30) * fivethous * even / (hundred + 1),
+ format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid,
+ format('%s/%s%s', odd, even, tenthous)::pg_lsn
+FROM tenk1 ORDER BY unique2 LIMIT 5 OFFSET 5;
+SELECT brin_desummarize_range('brinidx_bloom', 0);
+ brin_desummarize_range
+------------------------
+
+(1 row)
+
+VACUUM brintest_bloom; -- force a summarization cycle in brinidx
+UPDATE brintest_bloom SET int8col = int8col * int4col;
+UPDATE brintest_bloom SET textcol = '' WHERE textcol IS NOT NULL;
+-- Tests for brin_summarize_new_values
+SELECT brin_summarize_new_values('brintest_bloom'); -- error, not an index
+ERROR: "brintest_bloom" is not an index
+SELECT brin_summarize_new_values('tenk1_unique1'); -- error, not a BRIN index
+ERROR: "tenk1_unique1" is not a BRIN index
+SELECT brin_summarize_new_values('brinidx_bloom'); -- ok, no change expected
+ brin_summarize_new_values
+---------------------------
+ 0
+(1 row)
+
+-- Tests for brin_desummarize_range
+SELECT brin_desummarize_range('brinidx_bloom', -1); -- error, invalid range
+ERROR: block number out of range: -1
+SELECT brin_desummarize_range('brinidx_bloom', 0);
+ brin_desummarize_range
+------------------------
+
+(1 row)
+
+SELECT brin_desummarize_range('brinidx_bloom', 0);
+ brin_desummarize_range
+------------------------
+
+(1 row)
+
+SELECT brin_desummarize_range('brinidx_bloom', 100000000);
+ brin_desummarize_range
+------------------------
+
+(1 row)
+
+-- Test brin_summarize_range
+CREATE TABLE brin_summarize_bloom (
+ value int
+) WITH (fillfactor=10, autovacuum_enabled=false);
+CREATE INDEX brin_summarize_bloom_idx ON brin_summarize_bloom USING brin (value) WITH (pages_per_range=2);
+-- Fill a few pages
+DO $$
+DECLARE curtid tid;
+BEGIN
+ LOOP
+ INSERT INTO brin_summarize_bloom VALUES (1) RETURNING ctid INTO curtid;
+ EXIT WHEN curtid > tid '(2, 0)';
+ END LOOP;
+END;
+$$;
+-- summarize one range
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 0);
+ brin_summarize_range
+----------------------
+ 0
+(1 row)
+
+-- nothing: already summarized
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 1);
+ brin_summarize_range
+----------------------
+ 0
+(1 row)
+
+-- summarize one range
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 2);
+ brin_summarize_range
+----------------------
+ 1
+(1 row)
+
+-- nothing: page doesn't exist in table
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967295);
+ brin_summarize_range
+----------------------
+ 0
+(1 row)
+
+-- invalid block number values
+SELECT brin_summarize_range('brin_summarize_bloom_idx', -1);
+ERROR: block number out of range: -1
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967296);
+ERROR: block number out of range: 4294967296
+-- test brin cost estimates behave sanely based on correlation of values
+CREATE TABLE brin_test_bloom (a INT, b INT);
+INSERT INTO brin_test_bloom SELECT x/100,x%100 FROM generate_series(1,10000) x(x);
+CREATE INDEX brin_test_bloom_a_idx ON brin_test_bloom USING brin (a) WITH (pages_per_range = 2);
+CREATE INDEX brin_test_bloom_b_idx ON brin_test_bloom USING brin (b) WITH (pages_per_range = 2);
+VACUUM ANALYZE brin_test_bloom;
+-- Ensure brin index is used when columns are perfectly correlated
+EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE a = 1;
+ QUERY PLAN
+--------------------------------------------------
+ Bitmap Heap Scan on brin_test_bloom
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on brin_test_bloom_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+-- Ensure brin index is not used when values are not correlated
+EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE b = 1;
+ QUERY PLAN
+-----------------------------
+ Seq Scan on brin_test_bloom
+ Filter: (b = 1)
+(2 rows)
+
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3d..ef4b4444b9 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -2037,6 +2037,7 @@ ORDER BY 1, 2, 3;
2742 | 16 | @@
3580 | 1 | <
3580 | 1 | <<
+ 3580 | 1 | =
3580 | 2 | &<
3580 | 2 | <=
3580 | 3 | &&
@@ -2100,7 +2101,7 @@ ORDER BY 1, 2, 3;
4000 | 28 | ^@
4000 | 29 | <^
4000 | 30 | >^
-(123 rows)
+(124 rows)
-- Check that all opclass search operators have selectivity estimators.
-- This is not absolutely required, but it seems a reasonable thing
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..a7a5b22d4f 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4993,8 +4993,9 @@ List of access methods
List of operator classes
AM | Input type | Storage type | Operator class | Default?
------+------------+--------------+----------------+----------
+ brin | oid | | oid_bloom_ops | no
brin | oid | | oid_minmax_ops | yes
-(1 row)
+(2 rows)
\dAf spgist
List of operator families
diff --git a/src/test/regress/expected/type_sanity.out b/src/test/regress/expected/type_sanity.out
index 0c74dc96a8..e568b9fea2 100644
--- a/src/test/regress/expected/type_sanity.out
+++ b/src/test/regress/expected/type_sanity.out
@@ -67,13 +67,14 @@ WHERE p1.typtype not in ('p') AND p1.typname NOT LIKE E'\\_%'
WHERE p2.typname = ('_' || p1.typname)::name AND
p2.typelem = p1.oid and p1.typarray = p2.oid)
ORDER BY p1.oid;
- oid | typname
-------+-----------------
+ oid | typname
+------+-----------------------
194 | pg_node_tree
3361 | pg_ndistinct
3402 | pg_dependencies
5017 | pg_mcv_list
-(4 rows)
+ 9925 | pg_brin_bloom_summary
+(5 rows)
-- Make sure typarray points to a "true" array type of our own base
SELECT p1.oid, p1.typname as basetype, p2.typname as arraytype,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index c77b0d7342..ecd0806718 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -77,6 +77,11 @@ test: select_into select_distinct select_distinct_on select_implicit select_havi
# ----------
test: brin gin gist spgist privileges init_privs security_label collate matview lock replica_identity rowsecurity object_address tablesample groupingsets drop_operator password identity generated join_hash
+# ----------
+# Additional BRIN tests
+# ----------
+test: brin_bloom
+
# ----------
# Another group of parallel tests
# ----------
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0264a97324..c0d7fa76f1 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -108,6 +108,7 @@ test: delete
test: namespace
test: prepared_xacts
test: brin
+test: brin_bloom
test: gin
test: gist
test: spgist
diff --git a/src/test/regress/sql/brin_bloom.sql b/src/test/regress/sql/brin_bloom.sql
new file mode 100644
index 0000000000..5d499208e3
--- /dev/null
+++ b/src/test/regress/sql/brin_bloom.sql
@@ -0,0 +1,376 @@
+CREATE TABLE brintest_bloom (byteacol bytea,
+ charcol "char",
+ namecol name,
+ int8col bigint,
+ int2col smallint,
+ int4col integer,
+ textcol text,
+ oidcol oid,
+ float4col real,
+ float8col double precision,
+ macaddrcol macaddr,
+ inetcol inet,
+ cidrcol cidr,
+ bpcharcol character,
+ datecol date,
+ timecol time without time zone,
+ timestampcol timestamp without time zone,
+ timestamptzcol timestamp with time zone,
+ intervalcol interval,
+ timetzcol time with time zone,
+ numericcol numeric,
+ uuidcol uuid,
+ lsncol pg_lsn
+) WITH (fillfactor=10);
+
+INSERT INTO brintest_bloom SELECT
+ repeat(stringu1, 8)::bytea,
+ substr(stringu1, 1, 1)::"char",
+ stringu1::name, 142857 * tenthous,
+ thousand,
+ twothousand,
+ repeat(stringu1, 8),
+ unique1::oid,
+ (four + 1.0)/(hundred+1),
+ odd::float8 / (tenthous + 1),
+ format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr,
+ inet '10.2.3.4/24' + tenthous,
+ cidr '10.2.3/24' + tenthous,
+ substr(stringu1, 1, 1)::bpchar,
+ date '1995-08-15' + tenthous,
+ time '01:20:30' + thousand * interval '18.5 second',
+ timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours',
+ timestamptz '1972-10-10 03:00' + thousand * interval '1 hour',
+ justify_days(justify_hours(tenthous * interval '12 minutes')),
+ timetz '01:30:20+02' + hundred * interval '15 seconds',
+ tenthous::numeric(36,30) * fivethous * even / (hundred + 1),
+ format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid,
+ format('%s/%s%s', odd, even, tenthous)::pg_lsn
+FROM tenk1 ORDER BY unique2 LIMIT 100;
+
+-- throw in some NULL's and different values
+INSERT INTO brintest_bloom (inetcol, cidrcol) SELECT
+ inet 'fe80::6e40:8ff:fea9:8c46' + tenthous,
+ cidr 'fe80::6e40:8ff:fea9:8c46' + tenthous
+FROM tenk1 ORDER BY thousand, tenthous LIMIT 25;
+
+-- test bloom specific index options
+-- ndistinct must be >= -1.0
+CREATE INDEX brinidx_bloom ON brintest_bloom USING brin (
+ byteacol bytea_bloom_ops(n_distinct_per_range = -1.1)
+);
+-- false_positive_rate must be between 0.0001 and 0.25
+CREATE INDEX brinidx_bloom ON brintest_bloom USING brin (
+ byteacol bytea_bloom_ops(false_positive_rate = 0.00009)
+);
+CREATE INDEX brinidx_bloom ON brintest_bloom USING brin (
+ byteacol bytea_bloom_ops(false_positive_rate = 0.26)
+);
+
+CREATE INDEX brinidx_bloom ON brintest_bloom USING brin (
+ byteacol bytea_bloom_ops,
+ charcol char_bloom_ops,
+ namecol name_bloom_ops,
+ int8col int8_bloom_ops,
+ int2col int2_bloom_ops,
+ int4col int4_bloom_ops,
+ textcol text_bloom_ops,
+ oidcol oid_bloom_ops,
+ float4col float4_bloom_ops,
+ float8col float8_bloom_ops,
+ macaddrcol macaddr_bloom_ops,
+ inetcol inet_bloom_ops,
+ cidrcol inet_bloom_ops,
+ bpcharcol bpchar_bloom_ops,
+ datecol date_bloom_ops,
+ timecol time_bloom_ops,
+ timestampcol timestamp_bloom_ops,
+ timestamptzcol timestamptz_bloom_ops,
+ intervalcol interval_bloom_ops,
+ timetzcol timetz_bloom_ops,
+ numericcol numeric_bloom_ops,
+ uuidcol uuid_bloom_ops,
+ lsncol pg_lsn_bloom_ops
+) with (pages_per_range = 1);
+
+CREATE TABLE brinopers_bloom (colname name, typ text,
+ op text[], value text[], matches int[],
+ check (cardinality(op) = cardinality(value)),
+ check (cardinality(op) = cardinality(matches)));
+
+INSERT INTO brinopers_bloom VALUES
+ ('byteacol', 'bytea',
+ '{=}',
+ '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}',
+ '{1}'),
+ ('charcol', '"char"',
+ '{=}',
+ '{M}',
+ '{6}'),
+ ('namecol', 'name',
+ '{=}',
+ '{MAAAAA}',
+ '{2}'),
+ ('int2col', 'int2',
+ '{=}',
+ '{800}',
+ '{1}'),
+ ('int4col', 'int4',
+ '{=}',
+ '{800}',
+ '{1}'),
+ ('int8col', 'int8',
+ '{=}',
+ '{1257141600}',
+ '{1}'),
+ ('textcol', 'text',
+ '{=}',
+ '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}',
+ '{1}'),
+ ('oidcol', 'oid',
+ '{=}',
+ '{8800}',
+ '{1}'),
+ ('float4col', 'float4',
+ '{=}',
+ '{1}',
+ '{4}'),
+ ('float8col', 'float8',
+ '{=}',
+ '{0}',
+ '{1}'),
+ ('macaddrcol', 'macaddr',
+ '{=}',
+ '{2c:00:2d:00:16:00}',
+ '{2}'),
+ ('inetcol', 'inet',
+ '{=}',
+ '{10.2.14.231/24}',
+ '{1}'),
+ ('inetcol', 'cidr',
+ '{=}',
+ '{fe80::6e40:8ff:fea9:8c46}',
+ '{1}'),
+ ('cidrcol', 'inet',
+ '{=}',
+ '{10.2.14/24}',
+ '{2}'),
+ ('cidrcol', 'inet',
+ '{=}',
+ '{fe80::6e40:8ff:fea9:8c46}',
+ '{1}'),
+ ('cidrcol', 'cidr',
+ '{=}',
+ '{10.2.14/24}',
+ '{2}'),
+ ('cidrcol', 'cidr',
+ '{=}',
+ '{fe80::6e40:8ff:fea9:8c46}',
+ '{1}'),
+ ('bpcharcol', 'bpchar',
+ '{=}',
+ '{W}',
+ '{6}'),
+ ('datecol', 'date',
+ '{=}',
+ '{2009-12-01}',
+ '{1}'),
+ ('timecol', 'time',
+ '{=}',
+ '{02:28:57}',
+ '{1}'),
+ ('timestampcol', 'timestamp',
+ '{=}',
+ '{1964-03-24 19:26:45}',
+ '{1}'),
+ ('timestamptzcol', 'timestamptz',
+ '{=}',
+ '{1972-10-19 09:00:00-07}',
+ '{1}'),
+ ('intervalcol', 'interval',
+ '{=}',
+ '{1 mons 13 days 12:24}',
+ '{1}'),
+ ('timetzcol', 'timetz',
+ '{=}',
+ '{01:35:50+02}',
+ '{2}'),
+ ('numericcol', 'numeric',
+ '{=}',
+ '{2268164.347826086956521739130434782609}',
+ '{1}'),
+ ('uuidcol', 'uuid',
+ '{=}',
+ '{52225222-5222-5222-5222-522252225222}',
+ '{1}'),
+ ('lsncol', 'pg_lsn',
+ '{=, IS, IS NOT}',
+ '{44/455222, NULL, NULL}',
+ '{1, 25, 100}');
+
+DO $x$
+DECLARE
+ r record;
+ r2 record;
+ cond text;
+ idx_ctids tid[];
+ ss_ctids tid[];
+ count int;
+ plan_ok bool;
+ plan_line text;
+BEGIN
+ FOR r IN SELECT colname, oper, typ, value[ordinality], matches[ordinality] FROM brinopers_bloom, unnest(op) WITH ORDINALITY AS oper LOOP
+
+ -- prepare the condition
+ IF r.value IS NULL THEN
+ cond := format('%I %s %L', r.colname, r.oper, r.value);
+ ELSE
+ cond := format('%I %s %L::%s', r.colname, r.oper, r.value, r.typ);
+ END IF;
+
+ -- run the query using the brin index
+ SET enable_seqscan = 0;
+ SET enable_bitmapscan = 1;
+
+ plan_ok := false;
+ FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP
+ IF plan_line LIKE '%Bitmap Heap Scan on brintest_bloom%' THEN
+ plan_ok := true;
+ END IF;
+ END LOOP;
+ IF NOT plan_ok THEN
+ RAISE WARNING 'did not get bitmap indexscan plan for %', r;
+ END IF;
+
+ EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond)
+ INTO idx_ctids;
+
+ -- run the query using a seqscan
+ SET enable_seqscan = 1;
+ SET enable_bitmapscan = 0;
+
+ plan_ok := false;
+ FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP
+ IF plan_line LIKE '%Seq Scan on brintest_bloom%' THEN
+ plan_ok := true;
+ END IF;
+ END LOOP;
+ IF NOT plan_ok THEN
+ RAISE WARNING 'did not get seqscan plan for %', r;
+ END IF;
+
+ EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond)
+ INTO ss_ctids;
+
+ -- make sure both return the same results
+ count := array_length(idx_ctids, 1);
+
+ IF NOT (count = array_length(ss_ctids, 1) AND
+ idx_ctids @> ss_ctids AND
+ idx_ctids <@ ss_ctids) THEN
+ -- report the results of each scan to make the differences obvious
+ RAISE WARNING 'something not right in %: count %', r, count;
+ SET enable_seqscan = 1;
+ SET enable_bitmapscan = 0;
+ FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP
+ RAISE NOTICE 'seqscan: %', r2;
+ END LOOP;
+
+ SET enable_seqscan = 0;
+ SET enable_bitmapscan = 1;
+ FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP
+ RAISE NOTICE 'bitmapscan: %', r2;
+ END LOOP;
+ END IF;
+
+ -- make sure we found expected number of matches
+ IF count != r.matches THEN RAISE WARNING 'unexpected number of results % for %', count, r; END IF;
+ END LOOP;
+END;
+$x$;
+
+RESET enable_seqscan;
+RESET enable_bitmapscan;
+
+INSERT INTO brintest_bloom SELECT
+ repeat(stringu1, 42)::bytea,
+ substr(stringu1, 1, 1)::"char",
+ stringu1::name, 142857 * tenthous,
+ thousand,
+ twothousand,
+ repeat(stringu1, 42),
+ unique1::oid,
+ (four + 1.0)/(hundred+1),
+ odd::float8 / (tenthous + 1),
+ format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr,
+ inet '10.2.3.4' + tenthous,
+ cidr '10.2.3/24' + tenthous,
+ substr(stringu1, 1, 1)::bpchar,
+ date '1995-08-15' + tenthous,
+ time '01:20:30' + thousand * interval '18.5 second',
+ timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours',
+ timestamptz '1972-10-10 03:00' + thousand * interval '1 hour',
+ justify_days(justify_hours(tenthous * interval '12 minutes')),
+ timetz '01:30:20' + hundred * interval '15 seconds',
+ tenthous::numeric(36,30) * fivethous * even / (hundred + 1),
+ format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid,
+ format('%s/%s%s', odd, even, tenthous)::pg_lsn
+FROM tenk1 ORDER BY unique2 LIMIT 5 OFFSET 5;
+
+SELECT brin_desummarize_range('brinidx_bloom', 0);
+VACUUM brintest_bloom; -- force a summarization cycle in brinidx
+
+UPDATE brintest_bloom SET int8col = int8col * int4col;
+UPDATE brintest_bloom SET textcol = '' WHERE textcol IS NOT NULL;
+
+-- Tests for brin_summarize_new_values
+SELECT brin_summarize_new_values('brintest_bloom'); -- error, not an index
+SELECT brin_summarize_new_values('tenk1_unique1'); -- error, not a BRIN index
+SELECT brin_summarize_new_values('brinidx_bloom'); -- ok, no change expected
+
+-- Tests for brin_desummarize_range
+SELECT brin_desummarize_range('brinidx_bloom', -1); -- error, invalid range
+SELECT brin_desummarize_range('brinidx_bloom', 0);
+SELECT brin_desummarize_range('brinidx_bloom', 0);
+SELECT brin_desummarize_range('brinidx_bloom', 100000000);
+
+-- Test brin_summarize_range
+CREATE TABLE brin_summarize_bloom (
+ value int
+) WITH (fillfactor=10, autovacuum_enabled=false);
+CREATE INDEX brin_summarize_bloom_idx ON brin_summarize_bloom USING brin (value) WITH (pages_per_range=2);
+-- Fill a few pages
+DO $$
+DECLARE curtid tid;
+BEGIN
+ LOOP
+ INSERT INTO brin_summarize_bloom VALUES (1) RETURNING ctid INTO curtid;
+ EXIT WHEN curtid > tid '(2, 0)';
+ END LOOP;
+END;
+$$;
+
+-- summarize one range
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 0);
+-- nothing: already summarized
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 1);
+-- summarize one range
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 2);
+-- nothing: page doesn't exist in table
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967295);
+-- invalid block number values
+SELECT brin_summarize_range('brin_summarize_bloom_idx', -1);
+SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967296);
+
+
+-- test brin cost estimates behave sanely based on correlation of values
+CREATE TABLE brin_test_bloom (a INT, b INT);
+INSERT INTO brin_test_bloom SELECT x/100,x%100 FROM generate_series(1,10000) x(x);
+CREATE INDEX brin_test_bloom_a_idx ON brin_test_bloom USING brin (a) WITH (pages_per_range = 2);
+CREATE INDEX brin_test_bloom_b_idx ON brin_test_bloom USING brin (b) WITH (pages_per_range = 2);
+VACUUM ANALYZE brin_test_bloom;
+
+-- Ensure brin index is used when columns are perfectly correlated
+EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE a = 1;
+-- Ensure brin index is not used when values are not correlated
+EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE b = 1;
--
2.26.2
--------------AC3FFB734F56C5F52D422EFC
Content-Type: text/x-patch; charset=UTF-8;
name="0007-BRIN-minmax-multi-indexes-20210305.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="0007-BRIN-minmax-multi-indexes-20210305.patch"
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-16 08:55 Kyotaro Horiguchi <[email protected]>
0 siblings, 1 reply; 17+ messages in thread
From: Kyotaro Horiguchi @ 2023-02-16 08:55 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers
At Thu, 16 Feb 2023 06:20:23 +0000, "Hayato Kuroda (Fujitsu)" <[email protected]> wrote in
> Dear Horiguchi-san,
>
> Thank you for responding! Before modifying patches, I want to confirm something
> you said.
>
> > As Amit-K mentioned, we may need to change the name of the option in
> > this version, since the delay mechanism in this version causes a delay
> > in sending from publisher than delaying apply on the subscriber side.
>
> Right, will be changed.
>
> > I'm not sure why output plugin is involved in the delay mechanism. It
> > appears to me that it would be simpler if the delay occurred in
> > reorder buffer or logical decoder instead.
>
> I'm planning to change, but..
Yeah, I don't think we've made up our minds about which way to go yet,
so it's a bit too early to work on that.
> > Perhaps what I understand
> > correctly is that we could delay right before only sending commit
> > records in this case. If we delay at publisher end, all changes will
> > be sent at once if !streaming, and otherwise, all changes in a
> > transaction will be spooled at subscriber end. In any case, apply
> > worker won't be holding an active transaction unnecessarily.
>
> What about parallel case? Latest patch does not reject the combination of parallel
> streaming mode and delay. If delay is done at commit and subscriber uses an parallel
> apply worker, it may acquire lock for a long time.
I didn't looked too closely, but my guess is that transactions are
conveyed in spool files in parallel mode, with each file storing a
complete transaction.
> > Of
> > course we need add the mechanism to process keep-alive and status
> > report messages.
>
> Could you share the good way to handle keep-alive and status messages if you have?
> If we changed to the decoding layer, it is strange to call walsender function
> directly.
I'm sorry, but I don't have a concrete idea at the moment. When I read
through the last patch, I missed that WalSndDelay is actually a subset
of WalSndLoop. Although it can handle keep-alives correctly, I'm not
sure we can accept that structure..
> > Those setups work fine when no
> > apply-delay involved, but they won't work with the patches we're
> > talking about because the subscriber won't respond to the keep-alive
> > packets from the peer. So when I wrote "practically works" in the
> > last mail, this is what I meant.
>
> I'm not sure around the part. I think in the latest patch, subscriber can respond
> to the keepalive packets from the peer. Also, publisher can respond to the peer.
> Could you please tell me if you know a case that publisher or subscriber cannot
> respond to the opposite side? Note that if we apply the publisher-side patch, we
> don't have to apply subscriber-side patch.
Sorry about that again, I missed that part in the last patch as
mentioned earlier..
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-17 06:44 Hayato Kuroda (Fujitsu) <[email protected]>
parent: Kyotaro Horiguchi <[email protected]>
0 siblings, 1 reply; 17+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2023-02-17 06:44 UTC (permalink / raw)
To: 'Kyotaro Horiguchi' <[email protected]>; +Cc: [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Dear Horiguchi-san,
Thank you for replying! This direction seems OK, so I started to revise the patch.
PSA new version.
> > > As Amit-K mentioned, we may need to change the name of the option in
> > > this version, since the delay mechanism in this version causes a delay
> > > in sending from publisher than delaying apply on the subscriber side.
> >
> > Right, will be changed.
> >
> > > I'm not sure why output plugin is involved in the delay mechanism. It
> > > appears to me that it would be simpler if the delay occurred in
> > > reorder buffer or logical decoder instead.
> >
> > I'm planning to change, but..
>
> Yeah, I don't think we've made up our minds about which way to go yet,
> so it's a bit too early to work on that.
The parameter name is changed to min_send_delay.
And the delaying spot is changed to logical decoder.
> > > Perhaps what I understand
> > > correctly is that we could delay right before only sending commit
> > > records in this case. If we delay at publisher end, all changes will
> > > be sent at once if !streaming, and otherwise, all changes in a
> > > transaction will be spooled at subscriber end. In any case, apply
> > > worker won't be holding an active transaction unnecessarily.
> >
> > What about parallel case? Latest patch does not reject the combination of
> parallel
> > streaming mode and delay. If delay is done at commit and subscriber uses an
> parallel
> > apply worker, it may acquire lock for a long time.
>
> I didn't looked too closely, but my guess is that transactions are
> conveyed in spool files in parallel mode, with each file storing a
> complete transaction.
Based on the advice, I moved the delaying to DecodeCommit().
And the combination of parallel streaming mode and min_send_delay is
rejected again.
> > > Of
> > > course we need add the mechanism to process keep-alive and status
> > > report messages.
> >
> > Could you share the good way to handle keep-alive and status messages if you
> have?
> > If we changed to the decoding layer, it is strange to call walsender function
> > directly.
>
> I'm sorry, but I don't have a concrete idea at the moment. When I read
> through the last patch, I missed that WalSndDelay is actually a subset
> of WalSndLoop. Although it can handle keep-alives correctly, I'm not
> sure we can accept that structure..
No issues. I have kept the current implementation.
Some bugs I found are also fixed.
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
Attachments:
[application/octet-stream] v2-0001-Time-delayed-logical-replication-on-publisher-sid.patch (78.8K, ../../TYAPR01MB5866F00191375D0193320A4DF5A19@TYAPR01MB5866.jpnprd01.prod.outlook.com/2-v2-0001-Time-delayed-logical-replication-on-publisher-sid.patch)
download | inline diff:
From 2d119a5d5210ad3665e77646cff96e51e4c1e956 Mon Sep 17 00:00:00 2001
From: Hayato Kuroda <[email protected]>
Date: Thu, 16 Feb 2023 07:52:23 +0000
Subject: [PATCH v2] Time-delayed logical replication on publisher side
Similar to physical replication, a time-delayed copy of the data for
logical replication is useful for some scenarios (particularly to fix
errors that might cause data loss).
This patch implements a new subscription parameter called 'min_send_delay'.
If the subscription sets min_send_delay parameter, an apply worker passes the
value to the publisher as an output plugin option. And then, the walsender will
delay the transaction sending for given milliseconds.
The delay does not take into account the overhead of time spent in transferring
the transaction, which means that the arrival time at the subscriber may be
delayed more than the given time.
The combination of parallel streaming mode and min_send_delay is not allowed.
This is because in parallel streaming mode, we start applying the transaction
stream as soon as the first change arrives without knowing the transaction's
prepare/commit time. Always waiting for the full 'min_send_delay' period might
include unnecessary delay.
The other possibility was to apply the delay at the end of the parallel apply
transaction but that would cause issues related to resource bloat and locks being
held for a long time.
The delay occurs before we start to send the transaction on the publisher.
Regular and prepared transactions are covered. Streamed transactions are also
covered.
Author: Euler Taveira, Takamichi Osumi, Kuroda Hayato
Reviewed-by: Amit Kapila, Peter Smith, Vignesh C, Shveta Malik,
Kyotaro Horiguchi, Shi Yu, Wang Wei, Dilip Kumar, Melih Mutlu,
Andres Freund
---
doc/src/sgml/catalogs.sgml | 9 +
doc/src/sgml/glossary.sgml | 15 ++
doc/src/sgml/logical-replication.sgml | 6 +
doc/src/sgml/monitoring.sgml | 5 +
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 39 +++-
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 7 +-
src/backend/commands/subscriptioncmds.c | 122 +++++++++++-
.../libpqwalreceiver/libpqwalreceiver.c | 5 +
src/backend/replication/logical/decode.c | 4 +
src/backend/replication/logical/logical.c | 18 +-
.../replication/logical/logicalfuncs.c | 2 +-
src/backend/replication/logical/worker.c | 12 +-
src/backend/replication/pgoutput/pgoutput.c | 36 ++++
src/backend/replication/slotfuncs.c | 4 +-
src/backend/replication/walsender.c | 77 +++++++-
src/backend/utils/activity/wait_event.c | 3 +
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 9 +-
src/bin/psql/tab-complete.c | 4 +-
src/include/catalog/pg_subscription.h | 3 +
src/include/replication/logical.h | 13 +-
src/include/replication/pgoutput.h | 1 +
src/include/replication/walreceiver.h | 1 +
src/include/utils/wait_event.h | 3 +-
src/test/regress/expected/subscription.out | 174 ++++++++++--------
src/test/regress/sql/subscription.sql | 18 ++
src/test/subscription/t/001_rep_changes.pl | 28 +++
30 files changed, 536 insertions(+), 104 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c1e4048054..3c013f976a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -7873,6 +7873,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subminsenddelay</structfield> <type>int4</type>
+ </para>
+ <para>
+ The minimum delay for publisher sends data, in milliseconds
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subname</structfield> <type>name</type>
diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml
index 7c01a541fe..9ede9d05f6 100644
--- a/doc/src/sgml/glossary.sgml
+++ b/doc/src/sgml/glossary.sgml
@@ -1729,6 +1729,21 @@
</glossdef>
</glossentry>
+ <glossentry id="glossary-time-delayed-replication">
+ <glossterm>Time-delayed replication</glossterm>
+ <glossdef>
+ <para>
+ Replication setup that delays the application of changes by a specified
+ minimum time-delay period.
+ </para>
+ <para>
+ For more information, see
+ <xref linkend="guc-recovery-min-apply-delay"/> for physical replication
+ and <xref linkend="sql-createsubscription"/> for logical replication.
+ </para>
+ </glossdef>
+ </glossentry>
+
<glossentry id="glossary-toast">
<glossterm>TOAST</glossterm>
<glossdef>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 1bd5660c87..8bca0b3800 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -247,6 +247,12 @@
target table.
</para>
+ <para>
+ A publication can delay sending changes to the subscription by specifying
+ the <literal>min_send_delay</literal> subscription parameter. See
+ <xref linkend="sql-createsubscription"/> for details.
+ </para>
+
<sect2 id="logical-replication-subscription-slot">
<title>Replication Slot Management</title>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b0b997f092..e75525c5b8 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -2349,6 +2349,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry>Waiting to acquire an exclusive lock to truncate off any
empty pages at the end of a table vacuumed.</entry>
</row>
+ <row>
+ <entry><literal>WalSenderSendDelay</literal></entry>
+ <entry>Waiting for sending changes to subscriber in WAL sender
+ process.</entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 964fcbb8ff..3f238b958b 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -213,8 +213,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
are <literal>slot_name</literal>,
<literal>synchronous_commit</literal>,
<literal>binary</literal>, <literal>streaming</literal>,
- <literal>disable_on_error</literal>, and
- <literal>origin</literal>.
+ <literal>disable_on_error</literal>,
+ <literal>origin</literal>, and
+ <literal>min_send_delay</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 51c45f17c7..4a665c8d07 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -349,7 +349,39 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
- </variablelist></para>
+
+ <varlistentry>
+ <term><literal>min_send_delay</literal> (<type>integer</type>)</term>
+ <listitem>
+ <para>
+ By default, the publisher sends changes as soon as possible. This
+ parameter allows the user to delay the publisher to send changes by
+ given time period. If the value is specified without units, it is
+ taken as milliseconds. The default is zero (no delay). See
+ <xref linkend="config-setting-names-values"/> for details on the
+ available valid time units.
+ </para>
+ <para>
+ Any delay becomes effective only after all initial table
+ synchronization has finished and occurs before each transaction
+ starts to get applied on the subscriber. The delay does not take into
+ account the overhead of time spent in transferring the transaction,
+ which means that the arrival time at the subscriber may be delayed
+ more than the given time.
+ </para>
+ <warning>
+ <para>
+ Delaying the replication means there is a much longer time between
+ making a change on the publisher, and that change being committed
+ on the subscriber. This can impact the performance of synchronous
+ replication. See <xref linkend="guc-synchronous-commit"/>
+ parameter.
+ </para>
+ </warning>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
</listitem>
</varlistentry>
@@ -420,6 +452,11 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
published with different column lists are not supported.
</para>
+ <para>
+ A non-zero <literal>min_send_delay</literal> parameter is not allowed when
+ streaming in parallel mode.
+ </para>
+
<para>
We allow non-existent publications to be specified so that users can add
those later. This means
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index a56ae311c3..63a10b06d1 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -66,6 +66,7 @@ GetSubscription(Oid subid, bool missing_ok)
sub->skiplsn = subform->subskiplsn;
sub->name = pstrdup(NameStr(subform->subname));
sub->owner = subform->subowner;
+ sub->minsenddelay = subform->subminsenddelay;
sub->enabled = subform->subenabled;
sub->binary = subform->subbinary;
sub->stream = subform->substream;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 34ca0e739f..6b7b741a1e 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1314,9 +1314,10 @@ REVOKE ALL ON pg_replication_origin_status FROM public;
-- All columns of pg_subscription except subconninfo are publicly readable.
REVOKE ALL ON pg_subscription FROM public;
-GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
- subbinary, substream, subtwophasestate, subdisableonerr,
- subslotname, subsynccommit, subpublications, suborigin)
+GRANT SELECT (oid, subdbid, subskiplsn, subminsenddelay, subname, subowner,
+ subenabled, subbinary, substream, subtwophasestate,
+ subdisableonerr, subslotname, subsynccommit, subpublications,
+ suborigin)
ON pg_subscription TO public;
CREATE VIEW pg_stat_subscription_stats AS
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 464db6d247..fca23ae7e1 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -66,6 +66,7 @@
#define SUBOPT_DISABLE_ON_ERR 0x00000400
#define SUBOPT_LSN 0x00000800
#define SUBOPT_ORIGIN 0x00001000
+#define SUBOPT_MIN_SEND_DELAY 0x00002000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -90,6 +91,7 @@ typedef struct SubOpts
bool disableonerr;
char *origin;
XLogRecPtr lsn;
+ int32 min_send_delay;
} SubOpts;
static List *fetch_table_list(WalReceiverConn *wrconn, List *publications);
@@ -100,7 +102,7 @@ static void check_publications_origin(WalReceiverConn *wrconn,
static void check_duplicates_in_publist(List *publist, Datum *datums);
static List *merge_publications(List *oldpublist, List *newpublist, bool addpub, const char *subname);
static void ReportSlotConnectionError(List *rstates, Oid subid, char *slotname, char *err);
-
+static int32 defGetMinSendDelay(DefElem *def);
/*
* Common option parsing function for CREATE and ALTER SUBSCRIPTION commands.
@@ -146,6 +148,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->disableonerr = false;
if (IsSet(supported_opts, SUBOPT_ORIGIN))
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY))
+ opts->min_send_delay = 0;
/* Parse options */
foreach(lc, stmt_options)
@@ -324,6 +328,15 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->specified_opts |= SUBOPT_LSN;
opts->lsn = lsn;
}
+ else if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ if (IsSet(opts->specified_opts, SUBOPT_MIN_SEND_DELAY))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_MIN_SEND_DELAY;
+ opts->min_send_delay = defGetMinSendDelay(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -404,6 +417,32 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
"slot_name = NONE", "create_slot = false")));
}
}
+
+ /*
+ * The combination of parallel streaming mode and min_send_delay is not
+ * allowed. This is because in parallel streaming mode, we start applying
+ * the transaction stream as soon as the first change arrives without
+ * knowing the transaction's prepare/commit time. Always waiting for the
+ * full 'min_send_delay' period might include unnecessary delay.
+ *
+ * The other possibility was to apply the delay at the end of the parallel
+ * apply transaction but that would cause issues related to resource bloat
+ * and locks being held for a long time.
+ */
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ opts->min_send_delay > 0 &&
+ opts->streaming == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+
+ /*
+ * translator: the first %s is a string of the form "parameter > 0"
+ * and the second one is "option = value".
+ */
+ errmsg("%s and %s are mutually exclusive options",
+ "min_send_delay > 0", "streaming = parallel"));
+
+
}
/*
@@ -560,7 +599,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_SLOT_NAME | SUBOPT_COPY_DATA |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_TWOPHASE_COMMIT |
- SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN);
+ SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN |
+ SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -625,6 +665,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
values[Anum_pg_subscription_oid - 1] = ObjectIdGetDatum(subid);
values[Anum_pg_subscription_subdbid - 1] = ObjectIdGetDatum(MyDatabaseId);
values[Anum_pg_subscription_subskiplsn - 1] = LSNGetDatum(InvalidXLogRecPtr);
+ values[Anum_pg_subscription_subminsenddelay - 1] = Int32GetDatum(opts.min_send_delay);
values[Anum_pg_subscription_subname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->subname));
values[Anum_pg_subscription_subowner - 1] = ObjectIdGetDatum(owner);
@@ -1054,7 +1095,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
supported_opts = (SUBOPT_SLOT_NAME |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_DISABLE_ON_ERR |
- SUBOPT_ORIGIN);
+ SUBOPT_ORIGIN | SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options,
supported_opts, &opts);
@@ -1098,6 +1139,19 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
if (IsSet(opts.specified_opts, SUBOPT_STREAMING))
{
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.streaming == LOGICALREP_STREAM_PARALLEL &&
+ !IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY) &&
+ sub->minsenddelay > 0)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set parallel streaming mode for subscription with %s",
+ "min_send_delay"));
+
values[Anum_pg_subscription_substream - 1] =
CharGetDatum(opts.streaming);
replaces[Anum_pg_subscription_substream - 1] = true;
@@ -1111,6 +1165,26 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
= true;
}
+ if (IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY))
+ {
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.min_send_delay > 0 &&
+ !IsSet(opts.specified_opts, SUBOPT_STREAMING) &&
+ sub->stream == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set %s for subscription in parallel streaming mode",
+ "min_send_delay"));
+
+ values[Anum_pg_subscription_subminsenddelay - 1] =
+ Int32GetDatum(opts.min_send_delay);
+ replaces[Anum_pg_subscription_subminsenddelay - 1] = true;
+ }
+
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
{
values[Anum_pg_subscription_suborigin - 1] =
@@ -2195,3 +2269,45 @@ defGetStreamingMode(DefElem *def)
def->defname)));
return LOGICALREP_STREAM_OFF; /* keep compiler quiet */
}
+
+/*
+ * Extract the min_send_delay value from a DefElem. This is very similar to
+ * parse_and_validate_value() for integer values, because min_send_delay
+ * accepts the same parameter format as recovery_min_apply_delay.
+ */
+static int32
+defGetMinSendDelay(DefElem *def)
+{
+ char *input_string;
+ int result;
+ const char *hintmsg;
+
+ input_string = defGetString(def);
+
+ /*
+ * Parse given string as parameter which has millisecond unit
+ */
+ if (!parse_int(input_string, &result, GUC_UNIT_MS, &hintmsg))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid value for parameter \"%s\": \"%s\"",
+ "min_send_delay", input_string),
+ hintmsg ? errhint("%s", _(hintmsg)) : 0));
+
+ /*
+ * Check both the lower boundary for the valid min_send_delay range and
+ * the upper boundary as the safeguard for some platforms where INT_MAX is
+ * wider than int32 respectively. Although parse_int() has confirmed that
+ * the result is less than or equal to INT_MAX, the value will be stored
+ * in a catalog column of int32.
+ */
+ if (result < 0 || result > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%d ms is outside the valid range for parameter \"%s\" (%d .. %d)",
+ result,
+ "min_send_delay",
+ 0, PG_INT32_MAX)));
+
+ return result;
+}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 560ec974fa..89a72c1abe 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -443,6 +443,11 @@ libpqrcv_startstreaming(WalReceiverConn *conn,
PQserverVersion(conn->streamConn) >= 140000)
appendStringInfoString(&cmd, ", binary 'true'");
+ if (options->proto.logical.min_send_delay > 0 &&
+ PQserverVersion(conn->streamConn) >= 160000)
+ appendStringInfo(&cmd, ", min_send_delay '%d'",
+ options->proto.logical.min_send_delay);
+
appendStringInfoChar(&cmd, ')');
}
else
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index a53e23c679..80415baec4 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -666,6 +666,10 @@ DecodeCommit(LogicalDecodingContext *ctx, XLogRecordBuffer *buf,
buf->origptr, buf->endptr);
}
+ /* Delay given time if the context has 'delay' callback */
+ if (ctx->delay)
+ ctx->delay(ctx, commit_time);
+
/*
* Send the final commit record if the transaction data is already
* decoded, otherwise, process the entire transaction.
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c3ec97a0a6..ac1f9f92f7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -156,7 +156,8 @@ StartupDecodingContext(List *output_plugin_options,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay)
{
ReplicationSlot *slot;
MemoryContext context,
@@ -293,6 +294,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->prepare_write = prepare_write;
ctx->write = do_write;
ctx->update_progress = update_progress;
+ ctx->delay = delay;
ctx->output_plugin_options = output_plugin_options;
@@ -316,7 +318,7 @@ StartupDecodingContext(List *output_plugin_options,
* marking WAL reserved beforehand. In that scenario, it's up to the
* caller to guarantee that WAL remains available.
* xl_routine -- XLogReaderRoutine for underlying XLogReader
- * prepare_write, do_write, update_progress --
+ * prepare_write, do_write, update_progress, delay --
* callbacks that perform the use-case dependent, actual, work.
*
* Needs to be called while in a memory context that's at least as long lived
@@ -334,7 +336,8 @@ CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay)
{
TransactionId xmin_horizon = InvalidTransactionId;
ReplicationSlot *slot;
@@ -435,7 +438,7 @@ CreateInitDecodingContext(const char *plugin,
ctx = StartupDecodingContext(NIL, restart_lsn, xmin_horizon,
need_full_snapshot, false,
xl_routine, prepare_write, do_write,
- update_progress);
+ update_progress, delay);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
@@ -475,7 +478,7 @@ CreateInitDecodingContext(const char *plugin,
* xl_routine
* XLogReaderRoutine used by underlying xlogreader
*
- * prepare_write, do_write, update_progress
+ * prepare_write, do_write, update_progress, delay
* callbacks that have to be filled to perform the use-case dependent,
* actual work.
*
@@ -493,7 +496,8 @@ CreateDecodingContext(XLogRecPtr start_lsn,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay)
{
LogicalDecodingContext *ctx;
ReplicationSlot *slot;
@@ -547,7 +551,7 @@ CreateDecodingContext(XLogRecPtr start_lsn,
ctx = StartupDecodingContext(output_plugin_options,
start_lsn, InvalidTransactionId, false,
fast_forward, xl_routine, prepare_write,
- do_write, update_progress);
+ do_write, update_progress, delay);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c
index fa1b641a2b..960025197f 100644
--- a/src/backend/replication/logical/logicalfuncs.c
+++ b/src/backend/replication/logical/logicalfuncs.c
@@ -212,7 +212,7 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
LogicalOutputPrepareWrite,
- LogicalOutputWrite, NULL);
+ LogicalOutputWrite, NULL, NULL);
/*
* After the sanity checks in CreateDecodingContext, make sure the
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index cfb2ab6248..afbac3d80e 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -3898,7 +3898,8 @@ maybe_reread_subscription(void)
newsub->stream != MySubscription->stream ||
strcmp(newsub->origin, MySubscription->origin) != 0 ||
newsub->owner != MySubscription->owner ||
- !equal(newsub->publications, MySubscription->publications))
+ !equal(newsub->publications, MySubscription->publications) ||
+ newsub->minsenddelay != MySubscription->minsenddelay)
{
if (am_parallel_apply_worker())
ereport(LOG,
@@ -4617,9 +4618,18 @@ ApplyWorkerMain(Datum main_arg)
options.proto.logical.twophase = false;
options.proto.logical.origin = pstrdup(MySubscription->origin);
+ options.proto.logical.min_send_delay = 0;
if (!am_tablesync_worker())
{
+ /*
+ * Time-delayed logical replication does not support tablesync
+ * workers, so only the leader apply worker can request walsenders to
+ * apply delay on the publisher side.
+ */
+ if (server_version >= 160000 && MySubscription->minsenddelay > 0)
+ options.proto.logical.min_send_delay = MySubscription->minsenddelay;
+
/*
* Even when the two_phase mode is requested by the user, it remains
* as the tri-state PENDING until all tablesyncs have reached READY
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 98377c094b..be0095cf52 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -285,6 +285,7 @@ parse_output_parameters(List *options, PGOutputData *data)
bool streaming_given = false;
bool two_phase_option_given = false;
bool origin_option_given = false;
+ bool min_send_delay_option_given = false;
data->binary = false;
data->streaming = LOGICALREP_STREAM_OFF;
@@ -396,6 +397,32 @@ parse_output_parameters(List *options, PGOutputData *data)
errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized origin value: \"%s\"", data->origin));
}
+ else if (strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ long parsed;
+ char *endptr;
+
+ if (min_send_delay_option_given)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("conflicting or redundant options"));
+ min_send_delay_option_given = true;
+
+ errno = 0;
+ parsed = strtoul(strVal(defel->arg), &endptr, 10);
+ if (errno != 0 || *endptr != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid min_send_delay")));
+
+ if (parsed > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("min_send_delay \"%s\" out of range",
+ strVal(defel->arg))));
+
+ data->min_send_delay = (int32) parsed;
+ }
else
elog(ERROR, "unrecognized pgoutput option: %s", defel->defname);
}
@@ -501,6 +528,15 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
else
ctx->twophase_opt_given = true;
+ if (data->min_send_delay &&
+ data->protocol_version < LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("requested proto_version=%d does not support delay sending data, need %d or higher",
+ data->protocol_version, LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM)));
+ else
+ ctx->min_send_delay = data->min_send_delay;
+
/* Init publication state. */
data->publications = NIL;
publications_valid = false;
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 2f3c964824..522f7600a1 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -148,7 +148,7 @@ create_logical_replication_slot(char *name, char *plugin,
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* If caller needs us to determine the decoding start point, do so now.
@@ -481,7 +481,7 @@ pg_logical_replication_slot_advance(XLogRecPtr moveto)
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* Start reading at the slot's restart_lsn, which we know to point to
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 75e8363e24..a4f03ddba1 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -252,6 +252,7 @@ static void WalSndPrepareWrite(LogicalDecodingContext *ctx, XLogRecPtr lsn, Tran
static void WalSndWriteData(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid, bool last_write);
static void WalSndUpdateProgress(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid,
bool skipped_xact);
+static void WalSndDelay(LogicalDecodingContext *ctx, TimestampTz delay_start);
static XLogRecPtr WalSndWaitForWal(XLogRecPtr loc);
static void LagTrackerWrite(XLogRecPtr lsn, TimestampTz local_flush_time);
static TimeOffset LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now);
@@ -1126,7 +1127,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
/*
* Signal that we don't need the timeout mechanism. We're just
@@ -1285,7 +1286,7 @@ StartLogicalReplication(StartReplicationCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
xlogreader = logical_decoding_ctx->reader;
WalSndSetState(WALSNDSTATE_CATCHUP);
@@ -3849,3 +3850,75 @@ LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now)
Assert(time != 0);
return now - time;
}
+
+/*
+ * LogicalDecodingContext 'delay' callback.
+ *
+ * Wait long enough to make sure a transaction is applied at least that
+ * period behind the publisher.
+ */
+static void
+WalSndDelay(LogicalDecodingContext *ctx, TimestampTz delay_start)
+{
+ /* Wait till delayUntil by the latch mechanism */
+ while (true)
+ {
+ TimestampTz delayUntil;
+ long diffms;
+ long timeout_interval_ms;
+
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* This might change wal_sender_timeout */
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+
+ /* Check for input from the client */
+ ProcessRepliesIfAny();
+
+ /* die if timeout was reached */
+ WalSndCheckTimeOut();
+
+ /* Send keepalive if the time has come */
+ WalSndKeepaliveIfNecessary();
+
+ /* Try to flush pending output to the client */
+ if (pq_flush_if_writable() != 0)
+ WalSndShutdown();
+
+ /*
+ * If we've requested to shut down, exit the process.
+ *
+ * Note that WalSndDone() cannot be used here because the delaying
+ * changes will be sent in the function.
+ */
+ if (got_STOPPING)
+ WalSndShutdown();
+
+ delayUntil = TimestampTzPlusMilliseconds(delay_start, ctx->min_send_delay);
+ diffms = TimestampDifferenceMilliseconds(GetCurrentTimestamp(), delayUntil);
+
+ /*
+ * Exit without arming the latch if it's already past time to send
+ * this transaction.
+ */
+ if (diffms <= 0)
+ break;
+
+ /* Sleep until appropriate time. */
+ timeout_interval_ms = WalSndComputeSleeptime(GetCurrentTimestamp());
+
+ elog(DEBUG2, "time-delayed replication for txid %u, delay_time = %d ms, remaining wait time: %ld ms",
+ ctx->write_xid, (int) ctx->min_send_delay, diffms);
+
+ /* Sleep until we get reply from worker or we time out */
+ WalSndWait(WL_SOCKET_READABLE,
+ Min(timeout_interval_ms, diffms),
+ WAIT_EVENT_WALSENDER_SEND_DELAY);
+ }
+}
diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c
index cb99cc6339..76c19fe11d 100644
--- a/src/backend/utils/activity/wait_event.c
+++ b/src/backend/utils/activity/wait_event.c
@@ -515,6 +515,9 @@ pgstat_get_wait_timeout(WaitEventTimeout w)
case WAIT_EVENT_VACUUM_TRUNCATE:
event_name = "VacuumTruncate";
break;
+ case WAIT_EVENT_WALSENDER_SEND_DELAY:
+ event_name = "WalSenderSendDelay";
+ break;
/* no default case, so that compiler will warn */
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 527c7651ab..bd95747840 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4494,6 +4494,7 @@ getSubscriptions(Archive *fout)
int i_subsynccommit;
int i_subpublications;
int i_subbinary;
+ int i_subminsenddelay;
int i,
ntups;
@@ -4546,9 +4547,13 @@ getSubscriptions(Archive *fout)
LOGICALREP_TWOPHASE_STATE_DISABLED);
if (fout->remoteVersion >= 160000)
- appendPQExpBufferStr(query, " s.suborigin\n");
+ appendPQExpBufferStr(query,
+ " s.suborigin,\n"
+ " s.subminsenddelay\n");
else
- appendPQExpBuffer(query, " '%s' AS suborigin\n", LOGICALREP_ORIGIN_ANY);
+ appendPQExpBuffer(query, " '%s' AS suborigin,\n"
+ " 0 AS subminsenddelay\n",
+ LOGICALREP_ORIGIN_ANY);
appendPQExpBufferStr(query,
"FROM pg_subscription s\n"
@@ -4576,6 +4581,7 @@ getSubscriptions(Archive *fout)
i_subtwophasestate = PQfnumber(res, "subtwophasestate");
i_subdisableonerr = PQfnumber(res, "subdisableonerr");
i_suborigin = PQfnumber(res, "suborigin");
+ i_subminsenddelay = PQfnumber(res, "subminsenddelay");
subinfo = pg_malloc(ntups * sizeof(SubscriptionInfo));
@@ -4606,6 +4612,8 @@ getSubscriptions(Archive *fout)
subinfo[i].subdisableonerr =
pg_strdup(PQgetvalue(res, i, i_subdisableonerr));
subinfo[i].suborigin = pg_strdup(PQgetvalue(res, i, i_suborigin));
+ subinfo[i].subminsenddelay =
+ atoi(PQgetvalue(res, i, i_subminsenddelay));
/* Decide whether we want to dump it */
selectDumpableObject(&(subinfo[i].dobj), fout);
@@ -4687,6 +4695,9 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
+ if (subinfo->subminsenddelay > 0)
+ appendPQExpBuffer(query, ", min_send_delay = '%d ms'", subinfo->subminsenddelay);
+
appendPQExpBufferStr(query, ");\n");
if (subinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e7cbd8d7ed..24e0f6737f 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -661,6 +661,7 @@ typedef struct _SubscriptionInfo
char *subdisableonerr;
char *suborigin;
char *subsynccommit;
+ int subminsenddelay;
char *subpublications;
} SubscriptionInfo;
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index c8a0bb7b3a..c7d303a168 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6472,7 +6472,7 @@ describeSubscriptions(const char *pattern, bool verbose)
PGresult *res;
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
- false, false, false, false, false, false, false, false};
+ false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6527,10 +6527,13 @@ describeSubscriptions(const char *pattern, bool verbose)
gettext_noop("Two-phase commit"),
gettext_noop("Disable on error"));
+ /* Origin and min_send_delay are only supported in v16 and higher */
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
- ", suborigin AS \"%s\"\n",
- gettext_noop("Origin"));
+ ", suborigin AS \"%s\"\n"
+ ", subminsenddelay AS \"%s\"\n",
+ gettext_noop("Origin"),
+ gettext_noop("Min send delay"));
appendPQExpBuffer(&buf,
", subsynccommit AS \"%s\"\n"
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5e1882eaea..6643db6f55 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1925,7 +1925,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("(", "PUBLICATION");
/* ALTER SUBSCRIPTION <name> SET ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SET", "("))
- COMPLETE_WITH("binary", "disable_on_error", "origin", "slot_name",
+ COMPLETE_WITH("binary", "disable_on_error", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit");
/* ALTER SUBSCRIPTION <name> SKIP ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SKIP", "("))
@@ -3268,7 +3268,7 @@ psql_completion(const char *text, int start, int end)
/* Complete "CREATE SUBSCRIPTION <name> ... WITH ( <opt>" */
else if (HeadMatches("CREATE", "SUBSCRIPTION") && TailMatches("WITH", "("))
COMPLETE_WITH("binary", "connect", "copy_data", "create_slot",
- "disable_on_error", "enabled", "origin", "slot_name",
+ "disable_on_error", "enabled", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit", "two_phase");
/* CREATE TRIGGER --- is allowed inside CREATE SCHEMA, so use TailMatches */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index b0f2a1705d..69ae4314b4 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -74,6 +74,8 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
Oid subowner BKI_LOOKUP(pg_authid); /* Owner of the subscription */
+ int32 subminsenddelay; /* Replication send delay (ms) */
+
bool subenabled; /* True if the subscription is enabled (the
* worker should be running) */
@@ -122,6 +124,7 @@ typedef struct Subscription
* skipped */
char *name; /* Name of the subscription */
Oid owner; /* Oid of the subscription owner */
+ int32 minsenddelay; /* Replication send delay (ms) */
bool enabled; /* Indicates if the subscription is enabled */
bool binary; /* Indicates if the subscription wants data in
* binary format */
diff --git a/src/include/replication/logical.h b/src/include/replication/logical.h
index 5f49554ea0..603c37b6ce 100644
--- a/src/include/replication/logical.h
+++ b/src/include/replication/logical.h
@@ -30,6 +30,10 @@ typedef void (*LogicalOutputPluginWriterUpdateProgress) (struct LogicalDecodingC
bool skipped_xact
);
+typedef void (*LogicalOutputPluginWriterDelay) (struct LogicalDecodingContext *lr,
+ TimestampTz start_time
+);
+
typedef struct LogicalDecodingContext
{
/* memory context this is all allocated in */
@@ -64,6 +68,7 @@ typedef struct LogicalDecodingContext
LogicalOutputPluginWriterPrepareWrite prepare_write;
LogicalOutputPluginWriterWrite write;
LogicalOutputPluginWriterUpdateProgress update_progress;
+ LogicalOutputPluginWriterDelay delay;
/*
* Output buffer.
@@ -100,6 +105,8 @@ typedef struct LogicalDecodingContext
*/
bool twophase_opt_given;
+ int32 min_send_delay;
+
/*
* State for writing output.
*/
@@ -121,14 +128,16 @@ extern LogicalDecodingContext *CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay);
extern LogicalDecodingContext *CreateDecodingContext(XLogRecPtr start_lsn,
List *output_plugin_options,
bool fast_forward,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay);
extern void DecodingContextFindStartpoint(LogicalDecodingContext *ctx);
extern bool DecodingContextReady(LogicalDecodingContext *ctx);
extern void FreeDecodingContext(LogicalDecodingContext *ctx);
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index b4a8015403..d2fde09e00 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -30,6 +30,7 @@ typedef struct PGOutputData
bool messages;
bool two_phase;
char *origin;
+ int32 min_send_delay;
} PGOutputData;
#endif /* PGOUTPUT_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index decffe352d..c20969aed7 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -187,6 +187,7 @@ typedef struct
* prepare time */
char *origin; /* Only publish data originating from the
* specified origin */
+ int32 min_send_delay; /* The minimum send delay */
} logical;
} proto;
} WalRcvStreamOptions;
diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h
index 9ab23e1c4a..cc3a234eba 100644
--- a/src/include/utils/wait_event.h
+++ b/src/include/utils/wait_event.h
@@ -150,7 +150,8 @@ typedef enum
WAIT_EVENT_REGISTER_SYNC_REQUEST,
WAIT_EVENT_SPIN_DELAY,
WAIT_EVENT_VACUUM_DELAY,
- WAIT_EVENT_VACUUM_TRUNCATE
+ WAIT_EVENT_VACUUM_TRUNCATE,
+ WAIT_EVENT_WALSENDER_SEND_DELAY
} WaitEventTimeout;
/* ----------
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 3f99b14394..b14384e8e7 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -114,18 +114,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -143,10 +143,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -163,10 +163,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/12345
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/12345
(1 row)
-- ok - with lsn = NONE
@@ -175,10 +175,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
BEGIN;
@@ -210,10 +210,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
ERROR: invalid value for parameter "synchronous_commit": "foobar"
HINT: Available values: local, remote_write, remote_apply, on, off.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | local | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | local | dbname=regress_doesnotexist2 | 0/0
(1 row)
-- rename back to keep the rest simple
@@ -247,19 +247,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -271,27 +271,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication already exists
@@ -306,10 +306,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication used more than once
@@ -324,10 +324,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -363,10 +363,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
--fail - alter of two_phase option not supported.
@@ -375,10 +375,10 @@ ERROR: unrecognized subscription parameter: "two_phase"
-- but can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -388,10 +388,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -404,18 +404,48 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | 0 | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+ERROR: invalid value for parameter "min_send_delay": "foo"
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+ERROR: -1 ms is outside the valid range for parameter "min_send_delay" (0 .. 2147483647)
+-- fail - utilizing streaming = parallel with time-delayed replication is not supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+ERROR: min_send_delay > 0 and streaming = parallel are mutually exclusive options
+-- success -- min_send_delay value without unit is take as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 123 | off | dbname=regress_doesnotexit | 0/0
+(1 row)
+
+-- success -- min_send_delay value with unit is converted into ms and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 86400000 | off | dbname=regress_doesnotexit | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 7281f5fee2..2fae3b06c7 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -286,6 +286,24 @@ ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+
+-- fail - utilizing streaming = parallel with time-delayed replication is not supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+
+-- success -- min_send_delay value without unit is take as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+\dRs+
+
+-- success -- min_send_delay value with unit is converted into ms and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
RESET SESSION AUTHORIZATION;
DROP ROLE regress_subscription_user;
DROP ROLE regress_subscription_user2;
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 91aa068c95..8984e14d74 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -515,6 +515,34 @@ $node_publisher->poll_query_until('postgres',
or die
"Timed out while waiting for apply to restart after renaming SUBSCRIPTION";
+# Test time-delayed logical replication
+#
+# If the subscription sets min_send_delay parameter, the logical replication
+# worker will delay the transaction apply for min_send_delay milliseconds. We
+# verify this by looking at the time difference between a) when tuples are
+# inserted on the publisher, and b) when those changes are replicated on the
+# subscriber. Even on slow machines, this strategy will give predictable behavior.
+
+# Set min_send_delay parameter to 3 seconds
+my $delay = 3;
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub_renamed SET (min_send_delay = '${delay}s')");
+
+# Before doing the insertion, get the current timestamp that will be
+# used as a comparison base.
+my $publisher_insert_time = time();
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_ins VALUES (generate_series(1101, 1120))");
+
+# The publisher waits for the replication to complete
+$node_publisher->wait_for_catchup('tap_sub_renamed');
+
+# This test is successful if and only if the LSN has been applied with at least
+# the configured apply delay.
+ok( time() - $publisher_insert_time >= $delay,
+ "subscriber applies WAL only after replication delay for non-streaming transaction"
+);
+
# check all the cleanup
$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub_renamed");
--
2.27.0
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-18 05:48 Amit Kapila <[email protected]>
parent: Hayato Kuroda (Fujitsu) <[email protected]>
0 siblings, 1 reply; 17+ messages in thread
From: Amit Kapila @ 2023-02-18 05:48 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
On Fri, Feb 17, 2023 at 12:14 PM Hayato Kuroda (Fujitsu)
<[email protected]> wrote:
>
> Thank you for replying! This direction seems OK, so I started to revise the patch.
> PSA new version.
>
Few comments:
=============
1.
+ <para>
+ The minimum delay for publisher sends data, in milliseconds
+ </para></entry>
+ </row>
It would probably be better to write it as "The minimum delay, in
milliseconds, by the publisher to send changes"
2. The subminsenddelay is placed inconsistently in the patch. In the
docs (catalogs.sgml), system_views.sql, and in some places in the
code, it is after subskiplsn, but in the catalog table and
corresponding structure, it is placed after subowner. It should be
consistently placed after the subscription owner.
3.
+ <row>
+ <entry><literal>WalSenderSendDelay</literal></entry>
+ <entry>Waiting for sending changes to subscriber in WAL sender
+ process.</entry>
How about writing it as follows: "Waiting while sending changes for
time-delayed logical replication in the WAL sender process."?
4.
+ <para>
+ Any delay becomes effective only after all initial table
+ synchronization has finished and occurs before each transaction
+ starts to get applied on the subscriber. The delay does not take into
+ account the overhead of time spent in transferring the transaction,
+ which means that the arrival time at the subscriber may be delayed
+ more than the given time.
+ </para>
This needs to change based on a new approach. It should be something
like: "The delay is effective only when the publisher decides to send
a particular transaction downstream."
5.
+ * allowed. This is because in parallel streaming mode, we start applying
+ * the transaction stream as soon as the first change arrives without
+ * knowing the transaction's prepare/commit time. Always waiting for the
+ * full 'min_send_delay' period might include unnecessary delay.
+ *
+ * The other possibility was to apply the delay at the end of the parallel
+ * apply transaction but that would cause issues related to resource bloat
+ * and locks being held for a long time.
+ */
This part of the comments seems to imply more of a subscriber-side
delay approach. I think we should try to adjust these as per the
changed approach.
6.
@@ -666,6 +666,10 @@ DecodeCommit(LogicalDecodingContext *ctx,
XLogRecordBuffer *buf,
buf->origptr, buf->endptr);
}
+ /* Delay given time if the context has 'delay' callback */
+ if (ctx->delay)
+ ctx->delay(ctx, commit_time);
+
I think we should invoke delay functionality only when
ctx->min_send_delay > 0. Otherwise, there will be some unnecessary
overhead. We can change the comment along the lines of: "Delay sending
the changes if required. For streaming transactions, this means a
delay in sending the last stream but that is okay because on the
downstream the changes will be applied only after receiving the last
stream."
7. For 2PC transactions, I think we should add the delay in
DecodePrerpare. Because after receiving the PREPARE, the downstream
will apply the xact. In this case, we shouldn't add a delay for the
commit_prepared.
8.
+#
+# If the subscription sets min_send_delay parameter, the logical replication
+# worker will delay the transaction apply for min_send_delay milliseconds.
I think here also comments should be updated as per the changed
approach for applying the delay on the publisher side.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-20 02:27 Hayato Kuroda (Fujitsu) <[email protected]>
parent: Amit Kapila <[email protected]>
0 siblings, 2 replies; 17+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2023-02-20 02:27 UTC (permalink / raw)
To: 'Amit Kapila' <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Dear Amit,
Thank you for reviewing! PSA new version.
> 1.
> + <para>
> + The minimum delay for publisher sends data, in milliseconds
> + </para></entry>
> + </row>
>
> It would probably be better to write it as "The minimum delay, in
> milliseconds, by the publisher to send changes"
Fixed.
> 2. The subminsenddelay is placed inconsistently in the patch. In the
> docs (catalogs.sgml), system_views.sql, and in some places in the
> code, it is after subskiplsn, but in the catalog table and
> corresponding structure, it is placed after subowner. It should be
> consistently placed after the subscription owner.
Basically moved. Note that some parts were not changed like
maybe_reread_subscription() because the ordering had been already broken.
> 3.
> + <row>
> + <entry><literal>WalSenderSendDelay</literal></entry>
> + <entry>Waiting for sending changes to subscriber in WAL sender
> + process.</entry>
>
> How about writing it as follows: "Waiting while sending changes for
> time-delayed logical replication in the WAL sender process."?
Fixed.
> 4.
> + <para>
> + Any delay becomes effective only after all initial table
> + synchronization has finished and occurs before each transaction
> + starts to get applied on the subscriber. The delay does not take into
> + account the overhead of time spent in transferring the transaction,
> + which means that the arrival time at the subscriber may be delayed
> + more than the given time.
> + </para>
>
> This needs to change based on a new approach. It should be something
> like: "The delay is effective only when the publisher decides to send
> a particular transaction downstream."
Right, the first sentence is partially changed as you said.
> 5.
> + * allowed. This is because in parallel streaming mode, we start applying
> + * the transaction stream as soon as the first change arrives without
> + * knowing the transaction's prepare/commit time. Always waiting for the
> + * full 'min_send_delay' period might include unnecessary delay.
> + *
> + * The other possibility was to apply the delay at the end of the parallel
> + * apply transaction but that would cause issues related to resource bloat
> + * and locks being held for a long time.
> + */
>
> This part of the comments seems to imply more of a subscriber-side
> delay approach. I think we should try to adjust these as per the
> changed approach.
Adjusted.
> 6.
> @@ -666,6 +666,10 @@ DecodeCommit(LogicalDecodingContext *ctx,
> XLogRecordBuffer *buf,
> buf->origptr, buf->endptr);
> }
>
> + /* Delay given time if the context has 'delay' callback */
> + if (ctx->delay)
> + ctx->delay(ctx, commit_time);
> +
>
> I think we should invoke delay functionality only when
> ctx->min_send_delay > 0. Otherwise, there will be some unnecessary
> overhead. We can change the comment along the lines of: "Delay sending
> the changes if required. For streaming transactions, this means a
> delay in sending the last stream but that is okay because on the
> downstream the changes will be applied only after receiving the last
> stream."
Changed accordingly.
> 7. For 2PC transactions, I think we should add the delay in
> DecodePrerpare. Because after receiving the PREPARE, the downstream
> will apply the xact. In this case, we shouldn't add a delay for the
> commit_prepared.
Right, the transaction will be end when it receive PREPARE. Fixed.
I've tested locally and the delay seemed to be occurred at PREPARE phase.
> 8.
> +#
> +# If the subscription sets min_send_delay parameter, the logical replication
> +# worker will delay the transaction apply for min_send_delay milliseconds.
>
> I think here also comments should be updated as per the changed
> approach for applying the delay on the publisher side.
Fixed.
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
Attachments:
[application/octet-stream] v3-0001-Time-delayed-logical-replication-on-publisher-sid.patch (79.6K, ../../TYAPR01MB5866504C73ACCF98E0E09B11F5A49@TYAPR01MB5866.jpnprd01.prod.outlook.com/2-v3-0001-Time-delayed-logical-replication-on-publisher-sid.patch)
download | inline diff:
From 339413ed077c173eb61d12d5a4e4bcaab60e3178 Mon Sep 17 00:00:00 2001
From: Hayato Kuroda <[email protected]>
Date: Thu, 16 Feb 2023 07:52:23 +0000
Subject: [PATCH v3] Time-delayed logical replication on publisher side
Similar to physical replication, a time-delayed copy of the data for
logical replication is useful for some scenarios (particularly to fix
errors that might cause data loss).
This patch implements a new subscription parameter called 'min_send_delay'.
If the subscription sets min_send_delay parameter, an apply worker passes the
value to the publisher as an output plugin option. And then, the walsender will
delay the transaction sending for given milliseconds.
The delay does not take into account the overhead of time spent in transferring
the transaction, which means that the arrival time at the subscriber may be
delayed more than the given time.
The combination of parallel streaming mode and min_send_delay is not allowed.
This is because in parallel streaming mode, we start applying the transaction
stream as soon as the first change arrives without knowing the transaction's
prepare/commit time. Always waiting for the full 'min_send_delay' period might
include unnecessary delay.
The other possibility was to apply the delay at the end of the parallel apply
transaction but that would cause issues related to resource bloat and locks being
held for a long time.
The delay occurs before we start to send the transaction on the publisher.
Regular and prepared transactions are covered. Streamed transactions are also
covered.
Author: Euler Taveira, Takamichi Osumi, Kuroda Hayato
Reviewed-by: Amit Kapila, Peter Smith, Vignesh C, Shveta Malik,
Kyotaro Horiguchi, Shi Yu, Wang Wei, Dilip Kumar, Melih Mutlu,
Andres Freund
---
doc/src/sgml/catalogs.sgml | 9 +
doc/src/sgml/glossary.sgml | 15 ++
doc/src/sgml/logical-replication.sgml | 6 +
doc/src/sgml/monitoring.sgml | 5 +
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 39 +++-
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 7 +-
src/backend/commands/subscriptioncmds.c | 122 +++++++++++-
.../libpqwalreceiver/libpqwalreceiver.c | 5 +
src/backend/replication/logical/decode.c | 18 ++
src/backend/replication/logical/logical.c | 18 +-
.../replication/logical/logicalfuncs.c | 2 +-
src/backend/replication/logical/worker.c | 12 +-
src/backend/replication/pgoutput/pgoutput.c | 36 ++++
src/backend/replication/slotfuncs.c | 4 +-
src/backend/replication/walsender.c | 77 +++++++-
src/backend/utils/activity/wait_event.c | 3 +
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 9 +-
src/bin/psql/tab-complete.c | 4 +-
src/include/catalog/pg_subscription.h | 3 +
src/include/replication/logical.h | 13 +-
src/include/replication/pgoutput.h | 1 +
src/include/replication/walreceiver.h | 1 +
src/include/utils/wait_event.h | 3 +-
src/test/regress/expected/subscription.out | 174 ++++++++++--------
src/test/regress/sql/subscription.sql | 18 ++
src/test/subscription/t/001_rep_changes.pl | 28 +++
30 files changed, 550 insertions(+), 104 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c1e4048054..c41771d5ac 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -7892,6 +7892,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subminsenddelay</structfield> <type>int4</type>
+ </para>
+ <para>
+ The minimum delay, in milliseconds, by the publisher to send changes
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subenabled</structfield> <type>bool</type>
diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml
index 7c01a541fe..9ede9d05f6 100644
--- a/doc/src/sgml/glossary.sgml
+++ b/doc/src/sgml/glossary.sgml
@@ -1729,6 +1729,21 @@
</glossdef>
</glossentry>
+ <glossentry id="glossary-time-delayed-replication">
+ <glossterm>Time-delayed replication</glossterm>
+ <glossdef>
+ <para>
+ Replication setup that delays the application of changes by a specified
+ minimum time-delay period.
+ </para>
+ <para>
+ For more information, see
+ <xref linkend="guc-recovery-min-apply-delay"/> for physical replication
+ and <xref linkend="sql-createsubscription"/> for logical replication.
+ </para>
+ </glossdef>
+ </glossentry>
+
<glossentry id="glossary-toast">
<glossterm>TOAST</glossterm>
<glossdef>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 1bd5660c87..8bca0b3800 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -247,6 +247,12 @@
target table.
</para>
+ <para>
+ A publication can delay sending changes to the subscription by specifying
+ the <literal>min_send_delay</literal> subscription parameter. See
+ <xref linkend="sql-createsubscription"/> for details.
+ </para>
+
<sect2 id="logical-replication-subscription-slot">
<title>Replication Slot Management</title>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b0b997f092..6158587644 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -2349,6 +2349,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry>Waiting to acquire an exclusive lock to truncate off any
empty pages at the end of a table vacuumed.</entry>
</row>
+ <row>
+ <entry><literal>WalSenderSendDelay</literal></entry>
+ <entry>Waiting while sending changes for time-delayed logical replication
+ in the WAL sender process.</entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 964fcbb8ff..3f238b958b 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -213,8 +213,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
are <literal>slot_name</literal>,
<literal>synchronous_commit</literal>,
<literal>binary</literal>, <literal>streaming</literal>,
- <literal>disable_on_error</literal>, and
- <literal>origin</literal>.
+ <literal>disable_on_error</literal>,
+ <literal>origin</literal>, and
+ <literal>min_send_delay</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 51c45f17c7..9fd2922d69 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -349,7 +349,39 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
- </variablelist></para>
+
+ <varlistentry>
+ <term><literal>min_send_delay</literal> (<type>integer</type>)</term>
+ <listitem>
+ <para>
+ By default, the publisher sends changes as soon as possible. This
+ parameter allows the user to delay the publisher to send changes by
+ given time period. If the value is specified without units, it is
+ taken as milliseconds. The default is zero (no delay). See
+ <xref linkend="config-setting-names-values"/> for details on the
+ available valid time units.
+ </para>
+ <para>
+ The delay is effective only when the initial table synchronization
+ has been finished and the publisher decides to send a particular
+ transaction downstream. The delay does not take into account the
+ overhead of time spent in transferring the transaction, which means
+ that the arrival time at the subscriber may be delayed more than the
+ given time.
+ </para>
+ <warning>
+ <para>
+ Delaying the replication means there is a much longer time between
+ making a change on the publisher, and that change being committed
+ on the subscriber. This can impact the performance of synchronous
+ replication. See <xref linkend="guc-synchronous-commit"/>
+ parameter.
+ </para>
+ </warning>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
</listitem>
</varlistentry>
@@ -420,6 +452,11 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
published with different column lists are not supported.
</para>
+ <para>
+ A non-zero <literal>min_send_delay</literal> parameter is not allowed when
+ streaming in parallel mode.
+ </para>
+
<para>
We allow non-existent publications to be specified so that users can add
those later. This means
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index a56ae311c3..63a10b06d1 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -66,6 +66,7 @@ GetSubscription(Oid subid, bool missing_ok)
sub->skiplsn = subform->subskiplsn;
sub->name = pstrdup(NameStr(subform->subname));
sub->owner = subform->subowner;
+ sub->minsenddelay = subform->subminsenddelay;
sub->enabled = subform->subenabled;
sub->binary = subform->subbinary;
sub->stream = subform->substream;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 34ca0e739f..54a705d71b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1314,9 +1314,10 @@ REVOKE ALL ON pg_replication_origin_status FROM public;
-- All columns of pg_subscription except subconninfo are publicly readable.
REVOKE ALL ON pg_subscription FROM public;
-GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
- subbinary, substream, subtwophasestate, subdisableonerr,
- subslotname, subsynccommit, subpublications, suborigin)
+GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subminsenddelay,
+ subenabled, subbinary, substream, subtwophasestate,
+ subdisableonerr, subslotname, subsynccommit, subpublications,
+ suborigin)
ON pg_subscription TO public;
CREATE VIEW pg_stat_subscription_stats AS
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 464db6d247..951fa874e2 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -66,6 +66,7 @@
#define SUBOPT_DISABLE_ON_ERR 0x00000400
#define SUBOPT_LSN 0x00000800
#define SUBOPT_ORIGIN 0x00001000
+#define SUBOPT_MIN_SEND_DELAY 0x00002000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -90,6 +91,7 @@ typedef struct SubOpts
bool disableonerr;
char *origin;
XLogRecPtr lsn;
+ int32 min_send_delay;
} SubOpts;
static List *fetch_table_list(WalReceiverConn *wrconn, List *publications);
@@ -100,7 +102,7 @@ static void check_publications_origin(WalReceiverConn *wrconn,
static void check_duplicates_in_publist(List *publist, Datum *datums);
static List *merge_publications(List *oldpublist, List *newpublist, bool addpub, const char *subname);
static void ReportSlotConnectionError(List *rstates, Oid subid, char *slotname, char *err);
-
+static int32 defGetMinSendDelay(DefElem *def);
/*
* Common option parsing function for CREATE and ALTER SUBSCRIPTION commands.
@@ -146,6 +148,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->disableonerr = false;
if (IsSet(supported_opts, SUBOPT_ORIGIN))
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY))
+ opts->min_send_delay = 0;
/* Parse options */
foreach(lc, stmt_options)
@@ -324,6 +328,15 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->specified_opts |= SUBOPT_LSN;
opts->lsn = lsn;
}
+ else if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ if (IsSet(opts->specified_opts, SUBOPT_MIN_SEND_DELAY))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_MIN_SEND_DELAY;
+ opts->min_send_delay = defGetMinSendDelay(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -404,6 +417,32 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
"slot_name = NONE", "create_slot = false")));
}
}
+
+ /*
+ * The combination of parallel streaming mode and min_send_delay is not
+ * allowed. This is because in parallel streaming mode, the walsender
+ * starts sending the transaction stream without knowing the prepare/commit
+ * time of the transaction. Always waiting for the full 'min_send_delay'
+ * time to send may introduce unnecessary delay.
+ *
+ * The other possibility was to wait sending COMMIT record of the parallel
+ * apply transaction but that would cause issues related to resource bloat
+ * and locks being held for a long time.
+ */
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ opts->min_send_delay > 0 &&
+ opts->streaming == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+
+ /*
+ * translator: the first %s is a string of the form "parameter > 0"
+ * and the second one is "option = value".
+ */
+ errmsg("%s and %s are mutually exclusive options",
+ "min_send_delay > 0", "streaming = parallel"));
+
+
}
/*
@@ -560,7 +599,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_SLOT_NAME | SUBOPT_COPY_DATA |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_TWOPHASE_COMMIT |
- SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN);
+ SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN |
+ SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -628,6 +668,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
values[Anum_pg_subscription_subname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->subname));
values[Anum_pg_subscription_subowner - 1] = ObjectIdGetDatum(owner);
+ values[Anum_pg_subscription_subminsenddelay - 1] = Int32GetDatum(opts.min_send_delay);
values[Anum_pg_subscription_subenabled - 1] = BoolGetDatum(opts.enabled);
values[Anum_pg_subscription_subbinary - 1] = BoolGetDatum(opts.binary);
values[Anum_pg_subscription_substream - 1] = CharGetDatum(opts.streaming);
@@ -1054,7 +1095,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
supported_opts = (SUBOPT_SLOT_NAME |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_DISABLE_ON_ERR |
- SUBOPT_ORIGIN);
+ SUBOPT_ORIGIN | SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options,
supported_opts, &opts);
@@ -1098,6 +1139,19 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
if (IsSet(opts.specified_opts, SUBOPT_STREAMING))
{
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.streaming == LOGICALREP_STREAM_PARALLEL &&
+ !IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY) &&
+ sub->minsenddelay > 0)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set parallel streaming mode for subscription with %s",
+ "min_send_delay"));
+
values[Anum_pg_subscription_substream - 1] =
CharGetDatum(opts.streaming);
replaces[Anum_pg_subscription_substream - 1] = true;
@@ -1111,6 +1165,26 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
= true;
}
+ if (IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY))
+ {
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.min_send_delay > 0 &&
+ !IsSet(opts.specified_opts, SUBOPT_STREAMING) &&
+ sub->stream == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set %s for subscription in parallel streaming mode",
+ "min_send_delay"));
+
+ values[Anum_pg_subscription_subminsenddelay - 1] =
+ Int32GetDatum(opts.min_send_delay);
+ replaces[Anum_pg_subscription_subminsenddelay - 1] = true;
+ }
+
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
{
values[Anum_pg_subscription_suborigin - 1] =
@@ -2195,3 +2269,45 @@ defGetStreamingMode(DefElem *def)
def->defname)));
return LOGICALREP_STREAM_OFF; /* keep compiler quiet */
}
+
+/*
+ * Extract the min_send_delay value from a DefElem. This is very similar to
+ * parse_and_validate_value() for integer values, because min_send_delay
+ * accepts the same parameter format as recovery_min_apply_delay.
+ */
+static int32
+defGetMinSendDelay(DefElem *def)
+{
+ char *input_string;
+ int result;
+ const char *hintmsg;
+
+ input_string = defGetString(def);
+
+ /*
+ * Parse given string as parameter which has millisecond unit
+ */
+ if (!parse_int(input_string, &result, GUC_UNIT_MS, &hintmsg))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid value for parameter \"%s\": \"%s\"",
+ "min_send_delay", input_string),
+ hintmsg ? errhint("%s", _(hintmsg)) : 0));
+
+ /*
+ * Check both the lower boundary for the valid min_send_delay range and
+ * the upper boundary as the safeguard for some platforms where INT_MAX is
+ * wider than int32 respectively. Although parse_int() has confirmed that
+ * the result is less than or equal to INT_MAX, the value will be stored
+ * in a catalog column of int32.
+ */
+ if (result < 0 || result > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%d ms is outside the valid range for parameter \"%s\" (%d .. %d)",
+ result,
+ "min_send_delay",
+ 0, PG_INT32_MAX)));
+
+ return result;
+}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 560ec974fa..89a72c1abe 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -443,6 +443,11 @@ libpqrcv_startstreaming(WalReceiverConn *conn,
PQserverVersion(conn->streamConn) >= 140000)
appendStringInfoString(&cmd, ", binary 'true'");
+ if (options->proto.logical.min_send_delay > 0 &&
+ PQserverVersion(conn->streamConn) >= 160000)
+ appendStringInfo(&cmd, ", min_send_delay '%d'",
+ options->proto.logical.min_send_delay);
+
appendStringInfoChar(&cmd, ')');
}
else
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index a53e23c679..9816a536ab 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -679,6 +679,15 @@ DecodeCommit(LogicalDecodingContext *ctx, XLogRecordBuffer *buf,
}
else
{
+ /*
+ * Delay sending the changes if required. For streaming transactions,
+ * this means a delay in sending the last stream but that is OK because
+ * on the downstream the changes will be applied only after receiving
+ * the last stream.
+ */
+ if (ctx->min_send_delay > 0 && ctx->delay)
+ ctx->delay(ctx, commit_time);
+
ReorderBufferCommit(ctx->reorder, xid, buf->origptr, buf->endptr,
commit_time, origin_id, origin_lsn);
}
@@ -763,6 +772,15 @@ DecodePrepare(LogicalDecodingContext *ctx, XLogRecordBuffer *buf,
buf->origptr, buf->endptr);
}
+ /*
+ * Delay sending the changes if required. For streaming transactions,
+ * this means a delay in sending the last stream but that is OK because
+ * on the downstream the changes will be applied only after receiving
+ * the last stream.
+ */
+ if (ctx->min_send_delay > 0 && ctx->delay)
+ ctx->delay(ctx, prepare_time);
+
/* replay actions of all transaction + subtransactions in order */
ReorderBufferPrepare(ctx->reorder, xid, parsed->twophase_gid);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c3ec97a0a6..ac1f9f92f7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -156,7 +156,8 @@ StartupDecodingContext(List *output_plugin_options,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay)
{
ReplicationSlot *slot;
MemoryContext context,
@@ -293,6 +294,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->prepare_write = prepare_write;
ctx->write = do_write;
ctx->update_progress = update_progress;
+ ctx->delay = delay;
ctx->output_plugin_options = output_plugin_options;
@@ -316,7 +318,7 @@ StartupDecodingContext(List *output_plugin_options,
* marking WAL reserved beforehand. In that scenario, it's up to the
* caller to guarantee that WAL remains available.
* xl_routine -- XLogReaderRoutine for underlying XLogReader
- * prepare_write, do_write, update_progress --
+ * prepare_write, do_write, update_progress, delay --
* callbacks that perform the use-case dependent, actual, work.
*
* Needs to be called while in a memory context that's at least as long lived
@@ -334,7 +336,8 @@ CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay)
{
TransactionId xmin_horizon = InvalidTransactionId;
ReplicationSlot *slot;
@@ -435,7 +438,7 @@ CreateInitDecodingContext(const char *plugin,
ctx = StartupDecodingContext(NIL, restart_lsn, xmin_horizon,
need_full_snapshot, false,
xl_routine, prepare_write, do_write,
- update_progress);
+ update_progress, delay);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
@@ -475,7 +478,7 @@ CreateInitDecodingContext(const char *plugin,
* xl_routine
* XLogReaderRoutine used by underlying xlogreader
*
- * prepare_write, do_write, update_progress
+ * prepare_write, do_write, update_progress, delay
* callbacks that have to be filled to perform the use-case dependent,
* actual work.
*
@@ -493,7 +496,8 @@ CreateDecodingContext(XLogRecPtr start_lsn,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay)
{
LogicalDecodingContext *ctx;
ReplicationSlot *slot;
@@ -547,7 +551,7 @@ CreateDecodingContext(XLogRecPtr start_lsn,
ctx = StartupDecodingContext(output_plugin_options,
start_lsn, InvalidTransactionId, false,
fast_forward, xl_routine, prepare_write,
- do_write, update_progress);
+ do_write, update_progress, delay);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c
index fa1b641a2b..960025197f 100644
--- a/src/backend/replication/logical/logicalfuncs.c
+++ b/src/backend/replication/logical/logicalfuncs.c
@@ -212,7 +212,7 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
LogicalOutputPrepareWrite,
- LogicalOutputWrite, NULL);
+ LogicalOutputWrite, NULL, NULL);
/*
* After the sanity checks in CreateDecodingContext, make sure the
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index cfb2ab6248..afbac3d80e 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -3898,7 +3898,8 @@ maybe_reread_subscription(void)
newsub->stream != MySubscription->stream ||
strcmp(newsub->origin, MySubscription->origin) != 0 ||
newsub->owner != MySubscription->owner ||
- !equal(newsub->publications, MySubscription->publications))
+ !equal(newsub->publications, MySubscription->publications) ||
+ newsub->minsenddelay != MySubscription->minsenddelay)
{
if (am_parallel_apply_worker())
ereport(LOG,
@@ -4617,9 +4618,18 @@ ApplyWorkerMain(Datum main_arg)
options.proto.logical.twophase = false;
options.proto.logical.origin = pstrdup(MySubscription->origin);
+ options.proto.logical.min_send_delay = 0;
if (!am_tablesync_worker())
{
+ /*
+ * Time-delayed logical replication does not support tablesync
+ * workers, so only the leader apply worker can request walsenders to
+ * apply delay on the publisher side.
+ */
+ if (server_version >= 160000 && MySubscription->minsenddelay > 0)
+ options.proto.logical.min_send_delay = MySubscription->minsenddelay;
+
/*
* Even when the two_phase mode is requested by the user, it remains
* as the tri-state PENDING until all tablesyncs have reached READY
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 98377c094b..be0095cf52 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -285,6 +285,7 @@ parse_output_parameters(List *options, PGOutputData *data)
bool streaming_given = false;
bool two_phase_option_given = false;
bool origin_option_given = false;
+ bool min_send_delay_option_given = false;
data->binary = false;
data->streaming = LOGICALREP_STREAM_OFF;
@@ -396,6 +397,32 @@ parse_output_parameters(List *options, PGOutputData *data)
errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized origin value: \"%s\"", data->origin));
}
+ else if (strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ long parsed;
+ char *endptr;
+
+ if (min_send_delay_option_given)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("conflicting or redundant options"));
+ min_send_delay_option_given = true;
+
+ errno = 0;
+ parsed = strtoul(strVal(defel->arg), &endptr, 10);
+ if (errno != 0 || *endptr != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid min_send_delay")));
+
+ if (parsed > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("min_send_delay \"%s\" out of range",
+ strVal(defel->arg))));
+
+ data->min_send_delay = (int32) parsed;
+ }
else
elog(ERROR, "unrecognized pgoutput option: %s", defel->defname);
}
@@ -501,6 +528,15 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
else
ctx->twophase_opt_given = true;
+ if (data->min_send_delay &&
+ data->protocol_version < LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("requested proto_version=%d does not support delay sending data, need %d or higher",
+ data->protocol_version, LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM)));
+ else
+ ctx->min_send_delay = data->min_send_delay;
+
/* Init publication state. */
data->publications = NIL;
publications_valid = false;
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 2f3c964824..522f7600a1 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -148,7 +148,7 @@ create_logical_replication_slot(char *name, char *plugin,
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* If caller needs us to determine the decoding start point, do so now.
@@ -481,7 +481,7 @@ pg_logical_replication_slot_advance(XLogRecPtr moveto)
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* Start reading at the slot's restart_lsn, which we know to point to
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 75e8363e24..a4f03ddba1 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -252,6 +252,7 @@ static void WalSndPrepareWrite(LogicalDecodingContext *ctx, XLogRecPtr lsn, Tran
static void WalSndWriteData(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid, bool last_write);
static void WalSndUpdateProgress(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid,
bool skipped_xact);
+static void WalSndDelay(LogicalDecodingContext *ctx, TimestampTz delay_start);
static XLogRecPtr WalSndWaitForWal(XLogRecPtr loc);
static void LagTrackerWrite(XLogRecPtr lsn, TimestampTz local_flush_time);
static TimeOffset LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now);
@@ -1126,7 +1127,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
/*
* Signal that we don't need the timeout mechanism. We're just
@@ -1285,7 +1286,7 @@ StartLogicalReplication(StartReplicationCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
xlogreader = logical_decoding_ctx->reader;
WalSndSetState(WALSNDSTATE_CATCHUP);
@@ -3849,3 +3850,75 @@ LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now)
Assert(time != 0);
return now - time;
}
+
+/*
+ * LogicalDecodingContext 'delay' callback.
+ *
+ * Wait long enough to make sure a transaction is applied at least that
+ * period behind the publisher.
+ */
+static void
+WalSndDelay(LogicalDecodingContext *ctx, TimestampTz delay_start)
+{
+ /* Wait till delayUntil by the latch mechanism */
+ while (true)
+ {
+ TimestampTz delayUntil;
+ long diffms;
+ long timeout_interval_ms;
+
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* This might change wal_sender_timeout */
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+
+ /* Check for input from the client */
+ ProcessRepliesIfAny();
+
+ /* die if timeout was reached */
+ WalSndCheckTimeOut();
+
+ /* Send keepalive if the time has come */
+ WalSndKeepaliveIfNecessary();
+
+ /* Try to flush pending output to the client */
+ if (pq_flush_if_writable() != 0)
+ WalSndShutdown();
+
+ /*
+ * If we've requested to shut down, exit the process.
+ *
+ * Note that WalSndDone() cannot be used here because the delaying
+ * changes will be sent in the function.
+ */
+ if (got_STOPPING)
+ WalSndShutdown();
+
+ delayUntil = TimestampTzPlusMilliseconds(delay_start, ctx->min_send_delay);
+ diffms = TimestampDifferenceMilliseconds(GetCurrentTimestamp(), delayUntil);
+
+ /*
+ * Exit without arming the latch if it's already past time to send
+ * this transaction.
+ */
+ if (diffms <= 0)
+ break;
+
+ /* Sleep until appropriate time. */
+ timeout_interval_ms = WalSndComputeSleeptime(GetCurrentTimestamp());
+
+ elog(DEBUG2, "time-delayed replication for txid %u, delay_time = %d ms, remaining wait time: %ld ms",
+ ctx->write_xid, (int) ctx->min_send_delay, diffms);
+
+ /* Sleep until we get reply from worker or we time out */
+ WalSndWait(WL_SOCKET_READABLE,
+ Min(timeout_interval_ms, diffms),
+ WAIT_EVENT_WALSENDER_SEND_DELAY);
+ }
+}
diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c
index cb99cc6339..76c19fe11d 100644
--- a/src/backend/utils/activity/wait_event.c
+++ b/src/backend/utils/activity/wait_event.c
@@ -515,6 +515,9 @@ pgstat_get_wait_timeout(WaitEventTimeout w)
case WAIT_EVENT_VACUUM_TRUNCATE:
event_name = "VacuumTruncate";
break;
+ case WAIT_EVENT_WALSENDER_SEND_DELAY:
+ event_name = "WalSenderSendDelay";
+ break;
/* no default case, so that compiler will warn */
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 527c7651ab..bd95747840 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4494,6 +4494,7 @@ getSubscriptions(Archive *fout)
int i_subsynccommit;
int i_subpublications;
int i_subbinary;
+ int i_subminsenddelay;
int i,
ntups;
@@ -4546,9 +4547,13 @@ getSubscriptions(Archive *fout)
LOGICALREP_TWOPHASE_STATE_DISABLED);
if (fout->remoteVersion >= 160000)
- appendPQExpBufferStr(query, " s.suborigin\n");
+ appendPQExpBufferStr(query,
+ " s.suborigin,\n"
+ " s.subminsenddelay\n");
else
- appendPQExpBuffer(query, " '%s' AS suborigin\n", LOGICALREP_ORIGIN_ANY);
+ appendPQExpBuffer(query, " '%s' AS suborigin,\n"
+ " 0 AS subminsenddelay\n",
+ LOGICALREP_ORIGIN_ANY);
appendPQExpBufferStr(query,
"FROM pg_subscription s\n"
@@ -4576,6 +4581,7 @@ getSubscriptions(Archive *fout)
i_subtwophasestate = PQfnumber(res, "subtwophasestate");
i_subdisableonerr = PQfnumber(res, "subdisableonerr");
i_suborigin = PQfnumber(res, "suborigin");
+ i_subminsenddelay = PQfnumber(res, "subminsenddelay");
subinfo = pg_malloc(ntups * sizeof(SubscriptionInfo));
@@ -4606,6 +4612,8 @@ getSubscriptions(Archive *fout)
subinfo[i].subdisableonerr =
pg_strdup(PQgetvalue(res, i, i_subdisableonerr));
subinfo[i].suborigin = pg_strdup(PQgetvalue(res, i, i_suborigin));
+ subinfo[i].subminsenddelay =
+ atoi(PQgetvalue(res, i, i_subminsenddelay));
/* Decide whether we want to dump it */
selectDumpableObject(&(subinfo[i].dobj), fout);
@@ -4687,6 +4695,9 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
+ if (subinfo->subminsenddelay > 0)
+ appendPQExpBuffer(query, ", min_send_delay = '%d ms'", subinfo->subminsenddelay);
+
appendPQExpBufferStr(query, ");\n");
if (subinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e7cbd8d7ed..24e0f6737f 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -661,6 +661,7 @@ typedef struct _SubscriptionInfo
char *subdisableonerr;
char *suborigin;
char *subsynccommit;
+ int subminsenddelay;
char *subpublications;
} SubscriptionInfo;
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index c8a0bb7b3a..c7d303a168 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6472,7 +6472,7 @@ describeSubscriptions(const char *pattern, bool verbose)
PGresult *res;
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
- false, false, false, false, false, false, false, false};
+ false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6527,10 +6527,13 @@ describeSubscriptions(const char *pattern, bool verbose)
gettext_noop("Two-phase commit"),
gettext_noop("Disable on error"));
+ /* Origin and min_send_delay are only supported in v16 and higher */
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
- ", suborigin AS \"%s\"\n",
- gettext_noop("Origin"));
+ ", suborigin AS \"%s\"\n"
+ ", subminsenddelay AS \"%s\"\n",
+ gettext_noop("Origin"),
+ gettext_noop("Min send delay"));
appendPQExpBuffer(&buf,
", subsynccommit AS \"%s\"\n"
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5e1882eaea..6643db6f55 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1925,7 +1925,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("(", "PUBLICATION");
/* ALTER SUBSCRIPTION <name> SET ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SET", "("))
- COMPLETE_WITH("binary", "disable_on_error", "origin", "slot_name",
+ COMPLETE_WITH("binary", "disable_on_error", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit");
/* ALTER SUBSCRIPTION <name> SKIP ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SKIP", "("))
@@ -3268,7 +3268,7 @@ psql_completion(const char *text, int start, int end)
/* Complete "CREATE SUBSCRIPTION <name> ... WITH ( <opt>" */
else if (HeadMatches("CREATE", "SUBSCRIPTION") && TailMatches("WITH", "("))
COMPLETE_WITH("binary", "connect", "copy_data", "create_slot",
- "disable_on_error", "enabled", "origin", "slot_name",
+ "disable_on_error", "enabled", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit", "two_phase");
/* CREATE TRIGGER --- is allowed inside CREATE SCHEMA, so use TailMatches */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index b0f2a1705d..69ae4314b4 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -74,6 +74,8 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
Oid subowner BKI_LOOKUP(pg_authid); /* Owner of the subscription */
+ int32 subminsenddelay; /* Replication send delay (ms) */
+
bool subenabled; /* True if the subscription is enabled (the
* worker should be running) */
@@ -122,6 +124,7 @@ typedef struct Subscription
* skipped */
char *name; /* Name of the subscription */
Oid owner; /* Oid of the subscription owner */
+ int32 minsenddelay; /* Replication send delay (ms) */
bool enabled; /* Indicates if the subscription is enabled */
bool binary; /* Indicates if the subscription wants data in
* binary format */
diff --git a/src/include/replication/logical.h b/src/include/replication/logical.h
index 5f49554ea0..603c37b6ce 100644
--- a/src/include/replication/logical.h
+++ b/src/include/replication/logical.h
@@ -30,6 +30,10 @@ typedef void (*LogicalOutputPluginWriterUpdateProgress) (struct LogicalDecodingC
bool skipped_xact
);
+typedef void (*LogicalOutputPluginWriterDelay) (struct LogicalDecodingContext *lr,
+ TimestampTz start_time
+);
+
typedef struct LogicalDecodingContext
{
/* memory context this is all allocated in */
@@ -64,6 +68,7 @@ typedef struct LogicalDecodingContext
LogicalOutputPluginWriterPrepareWrite prepare_write;
LogicalOutputPluginWriterWrite write;
LogicalOutputPluginWriterUpdateProgress update_progress;
+ LogicalOutputPluginWriterDelay delay;
/*
* Output buffer.
@@ -100,6 +105,8 @@ typedef struct LogicalDecodingContext
*/
bool twophase_opt_given;
+ int32 min_send_delay;
+
/*
* State for writing output.
*/
@@ -121,14 +128,16 @@ extern LogicalDecodingContext *CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay);
extern LogicalDecodingContext *CreateDecodingContext(XLogRecPtr start_lsn,
List *output_plugin_options,
bool fast_forward,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay);
extern void DecodingContextFindStartpoint(LogicalDecodingContext *ctx);
extern bool DecodingContextReady(LogicalDecodingContext *ctx);
extern void FreeDecodingContext(LogicalDecodingContext *ctx);
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index b4a8015403..d2fde09e00 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -30,6 +30,7 @@ typedef struct PGOutputData
bool messages;
bool two_phase;
char *origin;
+ int32 min_send_delay;
} PGOutputData;
#endif /* PGOUTPUT_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index decffe352d..c20969aed7 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -187,6 +187,7 @@ typedef struct
* prepare time */
char *origin; /* Only publish data originating from the
* specified origin */
+ int32 min_send_delay; /* The minimum send delay */
} logical;
} proto;
} WalRcvStreamOptions;
diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h
index 9ab23e1c4a..cc3a234eba 100644
--- a/src/include/utils/wait_event.h
+++ b/src/include/utils/wait_event.h
@@ -150,7 +150,8 @@ typedef enum
WAIT_EVENT_REGISTER_SYNC_REQUEST,
WAIT_EVENT_SPIN_DELAY,
WAIT_EVENT_VACUUM_DELAY,
- WAIT_EVENT_VACUUM_TRUNCATE
+ WAIT_EVENT_VACUUM_TRUNCATE,
+ WAIT_EVENT_WALSENDER_SEND_DELAY
} WaitEventTimeout;
/* ----------
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 3f99b14394..b14384e8e7 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -114,18 +114,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -143,10 +143,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -163,10 +163,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/12345
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/12345
(1 row)
-- ok - with lsn = NONE
@@ -175,10 +175,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
BEGIN;
@@ -210,10 +210,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
ERROR: invalid value for parameter "synchronous_commit": "foobar"
HINT: Available values: local, remote_write, remote_apply, on, off.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | local | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | local | dbname=regress_doesnotexist2 | 0/0
(1 row)
-- rename back to keep the rest simple
@@ -247,19 +247,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -271,27 +271,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication already exists
@@ -306,10 +306,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication used more than once
@@ -324,10 +324,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -363,10 +363,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
--fail - alter of two_phase option not supported.
@@ -375,10 +375,10 @@ ERROR: unrecognized subscription parameter: "two_phase"
-- but can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -388,10 +388,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -404,18 +404,48 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | 0 | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+ERROR: invalid value for parameter "min_send_delay": "foo"
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+ERROR: -1 ms is outside the valid range for parameter "min_send_delay" (0 .. 2147483647)
+-- fail - utilizing streaming = parallel with time-delayed replication is not supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+ERROR: min_send_delay > 0 and streaming = parallel are mutually exclusive options
+-- success -- min_send_delay value without unit is take as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 123 | off | dbname=regress_doesnotexit | 0/0
+(1 row)
+
+-- success -- min_send_delay value with unit is converted into ms and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 86400000 | off | dbname=regress_doesnotexit | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 7281f5fee2..2fae3b06c7 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -286,6 +286,24 @@ ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+
+-- fail - utilizing streaming = parallel with time-delayed replication is not supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+
+-- success -- min_send_delay value without unit is take as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+\dRs+
+
+-- success -- min_send_delay value with unit is converted into ms and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
RESET SESSION AUTHORIZATION;
DROP ROLE regress_subscription_user;
DROP ROLE regress_subscription_user2;
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 91aa068c95..34807ffba2 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -515,6 +515,34 @@ $node_publisher->poll_query_until('postgres',
or die
"Timed out while waiting for apply to restart after renaming SUBSCRIPTION";
+# Test time-delayed logical replication
+#
+# If the subscription sets min_send_delay parameter, the walsender will delay
+# the transaction send for min_send_delay milliseconds. We verify this by
+# looking at the time difference between a) when tuples are inserted on the
+# publisher, and b) when those changes are replicated on the subscriber. Even
+# on slow machines, this strategy will give predictable behavior.
+
+# Set min_send_delay parameter to 3 seconds
+my $delay = 3;
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub_renamed SET (min_send_delay = '${delay}s')");
+
+# Before doing the insertion, get the current timestamp that will be
+# used as a comparison base.
+my $publisher_insert_time = time();
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_ins VALUES (generate_series(1101, 1120))");
+
+# The publisher waits for the replication to complete
+$node_publisher->wait_for_catchup('tap_sub_renamed');
+
+# This test is successful if and only if the LSN has been applied with at least
+# the configured apply delay.
+ok( time() - $publisher_insert_time >= $delay,
+ "subscriber applies WAL only after replication delay for non-streaming transaction"
+);
+
# check all the cleanup
$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub_renamed");
--
2.27.0
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-20 22:01 Peter Smith <[email protected]>
parent: Hayato Kuroda (Fujitsu) <[email protected]>
1 sibling, 2 replies; 17+ messages in thread
From: Peter Smith @ 2023-02-20 22:01 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Amit Kapila <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Here are some review comments for patch v3-0001.
(I haven't looked at the test code yet)
======
Commit Message
1.
If the subscription sets min_send_delay parameter, an apply worker passes the
value to the publisher as an output plugin option. And then, the walsender will
delay the transaction sending for given milliseconds.
~
1a.
"an apply worker" --> "the apply worker (via walrcv_startstreaming)".
~
1b.
"And then, the walsender" --> "The walsender"
~~~
2.
The combination of parallel streaming mode and min_send_delay is not allowed.
This is because in parallel streaming mode, we start applying the transaction
stream as soon as the first change arrives without knowing the transaction's
prepare/commit time. Always waiting for the full 'min_send_delay' period might
include unnecessary delay.
~
Is there another reason not to support this?
Even if streaming + min_send_delay incurs some extra delay, is that a
reason to reject outright the combination? What difference will the
potential of a few extra seconds overhead make when min_send_delay is
more likely to be far greater (e.g. minutes or hours)?
~~~
3.
The other possibility was to apply the delay at the end of the parallel apply
transaction but that would cause issues related to resource bloat and
locks being
held for a long time.
~
Is this explanation still relevant now you are doing pub-side delays?
======
doc/src/sgml/catalogs.sgml
4.
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subminsenddelay</structfield> <type>int4</type>
+ </para>
+ <para>
+ The minimum delay, in milliseconds, by the publisher to send changes
+ </para></entry>
+ </row>
"by the publisher to send changes" --> "by the publisher before sending changes"
======
doc/src/sgml/logical-replication.sgml
5.
+ <para>
+ A publication can delay sending changes to the subscription by specifying
+ the <literal>min_send_delay</literal> subscription parameter. See
+ <xref linkend="sql-createsubscription"/> for details.
+ </para>
~
This description seemed backwards because IIUC the PUBLICATION has
nothing to do with the delay really, the walsender is told what to do
by the SUBSCRIPTION. Anyway, this paragraph is in the "Subscriber"
section, so mentioning publications was a bit confusing.
SUGGESTION
A subscription can delay the receipt of changes by specifying the
min_send_delay subscription parameter. See ...
======
doc/src/sgml/monitoring.sgml
6.
+ <row>
+ <entry><literal>WalSenderSendDelay</literal></entry>
+ <entry>Waiting while sending changes for time-delayed logical replication
+ in the WAL sender process.</entry>
+ </row>
Should this say "Waiting before sending changes", instead of "Waiting
while sending changes"?
======
doc/src/sgml/ref/create_subscription.sgml
7.
+ <para>
+ By default, the publisher sends changes as soon as possible. This
+ parameter allows the user to delay the publisher to send changes by
+ given time period. If the value is specified without units, it is
+ taken as milliseconds. The default is zero (no delay). See
+ <xref linkend="config-setting-names-values"/> for details on the
+ available valid time units.
+ </para>
"to delay the publisher to send changes" --> "to delay changes"
~~~
8.
+ <para>
+ The delay is effective only when the initial table synchronization
+ has been finished and the publisher decides to send a particular
+ transaction downstream. The delay does not take into account the
+ overhead of time spent in transferring the transaction, which means
+ that the arrival time at the subscriber may be delayed more than the
+ given time.
+ </para>
I'm not sure about this mention about only "effective only when the
initial table synchronization has been finished"... Now that the delay
is pub-side I don't know that it is true anymore. The tablesync worker
will try to synchronize with the apply worker. IIUC during this
"synchronization" phase the apply worker might be getting delayed by
its own walsender, so therefore the tablesync might also be delayed
(due to syncing with the apply worker) won't it?
======
src/backend/commands/subscriptioncmds.c
9.
+ /*
+ * translator: the first %s is a string of the form "parameter > 0"
+ * and the second one is "option = value".
+ */
+ errmsg("%s and %s are mutually exclusive options",
+ "min_send_delay > 0", "streaming = parallel"));
+
+
}
Excessive whitespace.
======
src/backend/replication/logical/worker.c
10. ApplyWorkerMain
+ /*
+ * Time-delayed logical replication does not support tablesync
+ * workers, so only the leader apply worker can request walsenders to
+ * apply delay on the publisher side.
+ */
+ if (server_version >= 160000 && MySubscription->minsenddelay > 0)
+ options.proto.logical.min_send_delay = MySubscription->minsenddelay;
"apply delay" --> "delay"
======
src/backend/replication/pgoutput/pgoutput.c
11.
+ errno = 0;
+ parsed = strtoul(strVal(defel->arg), &endptr, 10);
+ if (errno != 0 || *endptr != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid min_send_delay")));
+
+ if (parsed > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("min_send_delay \"%s\" out of range",
+ strVal(defel->arg))));
Should the validation be also checking/asserting no negative numbers,
or actually should the min_send_delay be defined as a uint32 in the
first place?
~~~
12. pgoutput_startup
@@ -501,6 +528,15 @@ pgoutput_startup(LogicalDecodingContext *ctx,
OutputPluginOptions *opt,
else
ctx->twophase_opt_given = true;
+ if (data->min_send_delay &&
+ data->protocol_version < LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("requested proto_version=%d does not support delay sending
data, need %d or higher",
+ data->protocol_version, LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM)));
+ else
+ ctx->min_send_delay = data->min_send_delay;
IMO it doesn't make sense to compare this new feature with the
unrelated LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM protocol
version. I think we should define a new constant
LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM (even if it has the same
value as the LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM).
======
src/backend/replication/walsender.c
13. WalSndDelay
+ long diffms;
+ long timeout_interval_ms;
IMO some more informative name for these would make the code read better:
'diffms' --> 'remaining_wait_time_ms'
'timeout_interval_ms' --> 'timeout_sleeptime_ms'
~~~
14.
+ /* Sleep until we get reply from worker or we time out */
+ WalSndWait(WL_SOCKET_READABLE,
+ Min(timeout_interval_ms, diffms),
+ WAIT_EVENT_WALSENDER_SEND_DELAY);
Sorry, I didn't understand this comment "reply from worker"... AFAIK
here we are just sleeping, not waiting for replies from anywhere (???)
======
src/include/replication/logical.h
15.
@@ -64,6 +68,7 @@ typedef struct LogicalDecodingContext
LogicalOutputPluginWriterPrepareWrite prepare_write;
LogicalOutputPluginWriterWrite write;
LogicalOutputPluginWriterUpdateProgress update_progress;
+ LogicalOutputPluginWriterDelay delay;
~
15a.
Question: Is there some advantage to introducing another callback,
instead of just doing the delay inline?
~
15b.
Should this be a more informative member name like 'delay_send'?
~~~
16.
@@ -100,6 +105,8 @@ typedef struct LogicalDecodingContext
*/
bool twophase_opt_given;
+ int32 min_send_delay;
+
Missing comment for this new member.
------
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-21 01:06 Peter Smith <[email protected]>
parent: Hayato Kuroda (Fujitsu) <[email protected]>
1 sibling, 0 replies; 17+ messages in thread
From: Peter Smith @ 2023-02-21 01:06 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Amit Kapila <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Here are some review comments for the v3-0001 test code.
======
src/test/regress/sql/subscription.sql
1.
+-- fail - utilizing streaming = parallel with time-delayed
replication is not supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION
'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect =
false, streaming = parallel, min_send_delay = 123);
"utilizing" --> "specifying"
~~~
2.
+-- success -- min_send_delay value without unit is take as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION
'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect =
false, min_send_delay = 123);
+\dRs+
"without unit is take as" --> "without units is taken as"
~~~
3.
+-- success -- min_send_delay value with unit is converted into ms and
stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
"with unit is converted into ms" --> "with units other than ms is
converted to ms"
~~~
4. Missing tests?
Why have the previous ALTER SUBSCRIPTION tests been removed? AFAIK,
currently, there are no regression tests for error messages like:
test_sub=# ALTER SUBSCRIPTION sub1 SET (min_send_delay = 123);
ERROR: cannot set min_send_delay for subscription in parallel streaming mode
======
src/test/subscription/t/001_rep_changes.pl
5.
+# This test is successful if and only if the LSN has been applied with at least
+# the configured apply delay.
+ok( time() - $publisher_insert_time >= $delay,
+ "subscriber applies WAL only after replication delay for
non-streaming transaction"
+);
It's not strictly an "apply delay". Maybe this comment only needs to
say like below:
SUGGESTION
# This test is successful only if at least the configured delay has elapsed.
------
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-21 02:00 Amit Kapila <[email protected]>
parent: Peter Smith <[email protected]>
1 sibling, 1 reply; 17+ messages in thread
From: Amit Kapila @ 2023-02-21 02:00 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Hayato Kuroda (Fujitsu) <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
On Tue, Feb 21, 2023 at 3:31 AM Peter Smith <[email protected]> wrote:
>
>
> 2.
> The combination of parallel streaming mode and min_send_delay is not allowed.
> This is because in parallel streaming mode, we start applying the transaction
> stream as soon as the first change arrives without knowing the transaction's
> prepare/commit time. Always waiting for the full 'min_send_delay' period might
> include unnecessary delay.
>
> ~
>
> Is there another reason not to support this?
>
> Even if streaming + min_send_delay incurs some extra delay, is that a
> reason to reject outright the combination? What difference will the
> potential of a few extra seconds overhead make when min_send_delay is
> more likely to be far greater (e.g. minutes or hours)?
>
I think the point is that we don't know the commit time at the start
of streaming and even the transaction can be quite long in which case
adding the delay is not expected.
>
> ======
> doc/src/sgml/catalogs.sgml
>
> 4.
> + <row>
> + <entry role="catalog_table_entry"><para role="column_definition">
> + <structfield>subminsenddelay</structfield> <type>int4</type>
> + </para>
> + <para>
> + The minimum delay, in milliseconds, by the publisher to send changes
> + </para></entry>
> + </row>
>
> "by the publisher to send changes" --> "by the publisher before sending changes"
>
For the streaming (=on) case, we may end up sending changes before we
start to apply delay.
> ======
> doc/src/sgml/monitoring.sgml
>
> 6.
> + <row>
> + <entry><literal>WalSenderSendDelay</literal></entry>
> + <entry>Waiting while sending changes for time-delayed logical replication
> + in the WAL sender process.</entry>
> + </row>
>
> Should this say "Waiting before sending changes", instead of "Waiting
> while sending changes"?
>
In the streaming (non-parallel) case, we may have sent some changes
before wait as we wait only at commit/prepare time. The downstream
won't apply such changes till commit. So, this description makes sense
and this matches similar nearby descriptions.
>
> 8.
> + <para>
> + The delay is effective only when the initial table synchronization
> + has been finished and the publisher decides to send a particular
> + transaction downstream. The delay does not take into account the
> + overhead of time spent in transferring the transaction, which means
> + that the arrival time at the subscriber may be delayed more than the
> + given time.
> + </para>
>
> I'm not sure about this mention about only "effective only when the
> initial table synchronization has been finished"... Now that the delay
> is pub-side I don't know that it is true anymore.
>
This will still be true because we don't wait during the initial copy
(sync). The delay happens only when the replication starts.
> ======
> src/backend/commands/subscriptioncmds.c
> ======
> src/backend/replication/pgoutput/pgoutput.c
>
> 11.
> + errno = 0;
> + parsed = strtoul(strVal(defel->arg), &endptr, 10);
> + if (errno != 0 || *endptr != '\0')
> + ereport(ERROR,
> + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> + errmsg("invalid min_send_delay")));
> +
> + if (parsed > PG_INT32_MAX)
> + ereport(ERROR,
> + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> + errmsg("min_send_delay \"%s\" out of range",
> + strVal(defel->arg))));
>
> Should the validation be also checking/asserting no negative numbers,
> or actually should the min_send_delay be defined as a uint32 in the
> first place?
>
I don't see the need to change the datatype of min_send_delay as
compared to what we have min_apply_delay.
> ======
> src/include/replication/logical.h
>
> 15.
> @@ -64,6 +68,7 @@ typedef struct LogicalDecodingContext
> LogicalOutputPluginWriterPrepareWrite prepare_write;
> LogicalOutputPluginWriterWrite write;
> LogicalOutputPluginWriterUpdateProgress update_progress;
> + LogicalOutputPluginWriterDelay delay;
>
> ~
>
> 15a.
> Question: Is there some advantage to introducing another callback,
> instead of just doing the delay inline?
>
This is required because we need to check walsender's timeout and or
process replies during the delay.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-21 07:57 Hayato Kuroda (Fujitsu) <[email protected]>
parent: Peter Smith <[email protected]>
1 sibling, 1 reply; 17+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2023-02-21 07:57 UTC (permalink / raw)
To: 'Peter Smith' <[email protected]>; +Cc: Amit Kapila <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Dear Peter,
Thank you for reviewing! PSA new version.
> 1.
> If the subscription sets min_send_delay parameter, an apply worker passes the
> value to the publisher as an output plugin option. And then, the walsender will
> delay the transaction sending for given milliseconds.
>
> ~
>
> 1a.
> "an apply worker" --> "the apply worker (via walrcv_startstreaming)".
>
> ~
>
> 1b.
> "And then, the walsender" --> "The walsender"
Fixed.
> 2.
> The combination of parallel streaming mode and min_send_delay is not allowed.
> This is because in parallel streaming mode, we start applying the transaction
> stream as soon as the first change arrives without knowing the transaction's
> prepare/commit time. Always waiting for the full 'min_send_delay' period might
> include unnecessary delay.
>
> ~
>
> Is there another reason not to support this?
>
> Even if streaming + min_send_delay incurs some extra delay, is that a
> reason to reject outright the combination? What difference will the
> potential of a few extra seconds overhead make when min_send_delay is
> more likely to be far greater (e.g. minutes or hours)?
Another case I came up with is that streaming transactions are come continuously.
If there are many transactions to be streamed, the walsender must delay to send for
every transactions, for the given period. It means that arrival of transactions at
the subscriber may delay for approximately min_send_delay x # of transactions.
> 3.
> The other possibility was to apply the delay at the end of the parallel apply
> transaction but that would cause issues related to resource bloat and
> locks being
> held for a long time.
>
> ~
>
> Is this explanation still relevant now you are doing pub-side delays?
Slightly reworded. I think the problem may be occurred if we delay sending COMMIT
record for parallel applied transactions.
> doc/src/sgml/catalogs.sgml
>
> 4.
> + <row>
> + <entry role="catalog_table_entry"><para role="column_definition">
> + <structfield>subminsenddelay</structfield> <type>int4</type>
> + </para>
> + <para>
> + The minimum delay, in milliseconds, by the publisher to send changes
> + </para></entry>
> + </row>
>
> "by the publisher to send changes" --> "by the publisher before sending changes"
As Amit said[1], there is a possibility to delay after sending delay. So I changed to
"before sending COMMIT record". How do you think?
> doc/src/sgml/logical-replication.sgml
>
> 5.
> + <para>
> + A publication can delay sending changes to the subscription by specifying
> + the <literal>min_send_delay</literal> subscription parameter. See
> + <xref linkend="sql-createsubscription"/> for details.
> + </para>
>
> ~
>
> This description seemed backwards because IIUC the PUBLICATION has
> nothing to do with the delay really, the walsender is told what to do
> by the SUBSCRIPTION. Anyway, this paragraph is in the "Subscriber"
> section, so mentioning publications was a bit confusing.
>
> SUGGESTION
> A subscription can delay the receipt of changes by specifying the
> min_send_delay subscription parameter. See ...
Changed.
> doc/src/sgml/monitoring.sgml
>
> 6.
> + <row>
> + <entry><literal>WalSenderSendDelay</literal></entry>
> + <entry>Waiting while sending changes for time-delayed logical
> replication
> + in the WAL sender process.</entry>
> + </row>
>
> Should this say "Waiting before sending changes", instead of "Waiting
> while sending changes"?
Per discussion[1], I did not fix.
> doc/src/sgml/ref/create_subscription.sgml
>
> 7.
> + <para>
> + By default, the publisher sends changes as soon as possible. This
> + parameter allows the user to delay the publisher to send changes by
> + given time period. If the value is specified without units, it is
> + taken as milliseconds. The default is zero (no delay). See
> + <xref linkend="config-setting-names-values"/> for details on the
> + available valid time units.
> + </para>
>
> "to delay the publisher to send changes" --> "to delay changes"
Fixed.
> 8.
> + <para>
> + The delay is effective only when the initial table synchronization
> + has been finished and the publisher decides to send a particular
> + transaction downstream. The delay does not take into account the
> + overhead of time spent in transferring the transaction, which means
> + that the arrival time at the subscriber may be delayed more than the
> + given time.
> + </para>
>
> I'm not sure about this mention about only "effective only when the
> initial table synchronization has been finished"... Now that the delay
> is pub-side I don't know that it is true anymore. The tablesync worker
> will try to synchronize with the apply worker. IIUC during this
> "synchronization" phase the apply worker might be getting delayed by
> its own walsender, so therefore the tablesync might also be delayed
> (due to syncing with the apply worker) won't it?
I tested and checked codes. First of all, the tablesync worker request to send WALs
without min_send_delay, so changes will be sent and applied with no delays. In this meaning,
the table synchronization has not been affected by the feature. While checking,
however, there is a possibility that the state of table will be delayed to get
'readly' because the changing of status from SYNCDONE from READY is done by apply worker.
It may lead that two-phase will be delayed in getting to "enabled".
I added descriptions about it.
> src/backend/commands/subscriptioncmds.c
>
> 9.
> + /*
> + * translator: the first %s is a string of the form "parameter > 0"
> + * and the second one is "option = value".
> + */
> + errmsg("%s and %s are mutually exclusive options",
> + "min_send_delay > 0", "streaming = parallel"));
> +
> +
> }
>
> Excessive whitespace.
Adjusted.
> src/backend/replication/logical/worker.c
>
> 10. ApplyWorkerMain
>
> + /*
> + * Time-delayed logical replication does not support tablesync
> + * workers, so only the leader apply worker can request walsenders to
> + * apply delay on the publisher side.
> + */
> + if (server_version >= 160000 && MySubscription->minsenddelay > 0)
> + options.proto.logical.min_send_delay = MySubscription->minsenddelay;
>
> "apply delay" --> "delay"
Fixed.
> src/backend/replication/pgoutput/pgoutput.c
>
> 11.
> + errno = 0;
> + parsed = strtoul(strVal(defel->arg), &endptr, 10);
> + if (errno != 0 || *endptr != '\0')
> + ereport(ERROR,
> + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> + errmsg("invalid min_send_delay")));
> +
> + if (parsed > PG_INT32_MAX)
> + ereport(ERROR,
> + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> + errmsg("min_send_delay \"%s\" out of range",
> + strVal(defel->arg))));
>
> Should the validation be also checking/asserting no negative numbers,
> or actually should the min_send_delay be defined as a uint32 in the
> first place?
I think you are right because min_apply_delay does not have related code.
we must consider additional possibility that user may send START_REPLICATION
by hand and it has minus value.
Fixed.
> 12. pgoutput_startup
>
> @@ -501,6 +528,15 @@ pgoutput_startup(LogicalDecodingContext *ctx,
> OutputPluginOptions *opt,
> else
> ctx->twophase_opt_given = true;
>
> + if (data->min_send_delay &&
> + data->protocol_version <
> LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM)
> + ereport(ERROR,
> + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> + errmsg("requested proto_version=%d does not support delay sending
> data, need %d or higher",
> + data->protocol_version,
> LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM)));
> + else
> + ctx->min_send_delay = data->min_send_delay;
>
>
> IMO it doesn't make sense to compare this new feature with the
> unrelated LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM protocol
> version. I think we should define a new constant
> LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM (even if it has the
> same
> value as the LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM).
Added.
> src/backend/replication/walsender.c
>
> 13. WalSndDelay
>
> + long diffms;
> + long timeout_interval_ms;
>
> IMO some more informative name for these would make the code read better:
>
> 'diffms' --> 'remaining_wait_time_ms'
> 'timeout_interval_ms' --> 'timeout_sleeptime_ms'
Changed.
> 14.
> + /* Sleep until we get reply from worker or we time out */
> + WalSndWait(WL_SOCKET_READABLE,
> + Min(timeout_interval_ms, diffms),
> + WAIT_EVENT_WALSENDER_SEND_DELAY);
>
> Sorry, I didn't understand this comment "reply from worker"... AFAIK
> here we are just sleeping, not waiting for replies from anywhere (???)
>
> ======
> src/include/replication/logical.h
>
> 15.
> @@ -64,6 +68,7 @@ typedef struct LogicalDecodingContext
> LogicalOutputPluginWriterPrepareWrite prepare_write;
> LogicalOutputPluginWriterWrite write;
> LogicalOutputPluginWriterUpdateProgress update_progress;
> + LogicalOutputPluginWriterDelay delay;
>
> ~
>
> 15a.
> Question: Is there some advantage to introducing another callback,
> instead of just doing the delay inline?
IIUC functions related with walsender should not be called directly, because there
is a possibility that replication slots are manipulated from the backed.
> 15b.
> Should this be a more informative member name like 'delay_send'?
Changed.
> 16.
> @@ -100,6 +105,8 @@ typedef struct LogicalDecodingContext
> */
> bool twophase_opt_given;
>
> + int32 min_send_delay;
> +
>
> Missing comment for this new member.
Added.
[1]: https://www.postgresql.org/message-id/[email protected]...
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
Attachments:
[application/octet-stream] v4-0001-Time-delayed-logical-replication-on-publisher-sid.patch (82.4K, ../../TYAPR01MB5866C6BCA4D9386D9C486033F5A59@TYAPR01MB5866.jpnprd01.prod.outlook.com/2-v4-0001-Time-delayed-logical-replication-on-publisher-sid.patch)
download | inline diff:
From 0239a31307a9036236bd3342eb50a5717ae9349f Mon Sep 17 00:00:00 2001
From: Hayato Kuroda <[email protected]>
Date: Thu, 16 Feb 2023 07:52:23 +0000
Subject: [PATCH v4] Time-delayed logical replication on publisher side
Similar to physical replication, a time-delayed copy of the data for
logical replication is useful for some scenarios (particularly to fix
errors that might cause data loss).
This patch implements a new subscription parameter called 'min_send_delay'.
If the subscription sets min_send_delay parameter, the apply worker
(via walrcv_startstreaming) passes the value to the publisher as an output plugin
option. The walsender will delay the transaction sending for given milliseconds.
The delay does not take into account the overhead of time spent in transferring
the transaction, which means that the arrival time at the subscriber may be
delayed more than the given time.
The combination of parallel streaming mode and min_send_delay is not allowed.
This is because in parallel streaming mode, we start applying the transaction
stream as soon as the first change arrives without knowing the transaction's
prepare/commit time. Always waiting for the full 'min_send_delay' period might
include unnecessary delay.
The other possibility was to apply the delay at the end of the parallel apply
transaction but that would cause issues related to resource bloat and locks being
held for a long time.
The delay occurs before we start to send the transaction on the publisher.
Regular and prepared transactions are covered. Streamed transactions are also
covered.
Eariler versions were written by Euler Taveira, Takamichi Osumi, and Kuroda Hayato
Author: Kuroda Hayato, Takamichi Osumi, Kuroda Hayato
Reviewed-by: Amit Kapila, Peter Smith, Vignesh C, Shveta Malik,
Kyotaro Horiguchi, Shi Yu, Wang Wei, Dilip Kumar, Melih Mutlu,
Andres Freund
---
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/glossary.sgml | 15 ++
doc/src/sgml/logical-replication.sgml | 6 +
doc/src/sgml/monitoring.sgml | 5 +
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 43 +++-
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 7 +-
src/backend/commands/subscriptioncmds.c | 119 ++++++++++-
.../libpqwalreceiver/libpqwalreceiver.c | 5 +
src/backend/replication/logical/decode.c | 18 ++
src/backend/replication/logical/logical.c | 18 +-
.../replication/logical/logicalfuncs.c | 2 +-
src/backend/replication/logical/worker.c | 12 +-
src/backend/replication/pgoutput/pgoutput.c | 36 ++++
src/backend/replication/slotfuncs.c | 4 +-
src/backend/replication/walsender.c | 78 +++++++-
src/backend/utils/activity/wait_event.c | 3 +
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 9 +-
src/bin/psql/tab-complete.c | 4 +-
src/include/catalog/pg_subscription.h | 3 +
src/include/replication/logical.h | 17 +-
src/include/replication/logicalproto.h | 6 +-
src/include/replication/pgoutput.h | 1 +
src/include/replication/walreceiver.h | 1 +
src/include/utils/wait_event.h | 3 +-
src/test/regress/expected/subscription.out | 185 +++++++++++-------
src/test/regress/sql/subscription.sql | 29 +++
src/test/subscription/t/001_rep_changes.pl | 27 +++
31 files changed, 583 insertions(+), 105 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c1e4048054..ee4c3c77e6 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -7892,6 +7892,16 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subminsenddelay</structfield> <type>int4</type>
+ </para>
+ <para>
+ The minimum delay, in milliseconds, by the publisher before sending
+ COMMIT record
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subenabled</structfield> <type>bool</type>
diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml
index 7c01a541fe..9ede9d05f6 100644
--- a/doc/src/sgml/glossary.sgml
+++ b/doc/src/sgml/glossary.sgml
@@ -1729,6 +1729,21 @@
</glossdef>
</glossentry>
+ <glossentry id="glossary-time-delayed-replication">
+ <glossterm>Time-delayed replication</glossterm>
+ <glossdef>
+ <para>
+ Replication setup that delays the application of changes by a specified
+ minimum time-delay period.
+ </para>
+ <para>
+ For more information, see
+ <xref linkend="guc-recovery-min-apply-delay"/> for physical replication
+ and <xref linkend="sql-createsubscription"/> for logical replication.
+ </para>
+ </glossdef>
+ </glossentry>
+
<glossentry id="glossary-toast">
<glossterm>TOAST</glossterm>
<glossdef>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 1bd5660c87..fe9e7f7b26 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -247,6 +247,12 @@
target table.
</para>
+ <para>
+ A subscription can delay the receipt of changes by specifying the
+ <literal>min_send_delay</literal> subscription parameter. See
+ <xref linkend="sql-createsubscription"/> for details.
+ </para>
+
<sect2 id="logical-replication-subscription-slot">
<title>Replication Slot Management</title>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b0b997f092..6158587644 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -2349,6 +2349,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry>Waiting to acquire an exclusive lock to truncate off any
empty pages at the end of a table vacuumed.</entry>
</row>
+ <row>
+ <entry><literal>WalSenderSendDelay</literal></entry>
+ <entry>Waiting while sending changes for time-delayed logical replication
+ in the WAL sender process.</entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 964fcbb8ff..3f238b958b 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -213,8 +213,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
are <literal>slot_name</literal>,
<literal>synchronous_commit</literal>,
<literal>binary</literal>, <literal>streaming</literal>,
- <literal>disable_on_error</literal>, and
- <literal>origin</literal>.
+ <literal>disable_on_error</literal>,
+ <literal>origin</literal>, and
+ <literal>min_send_delay</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 51c45f17c7..3862388e87 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -349,7 +349,43 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
- </variablelist></para>
+
+ <varlistentry>
+ <term><literal>min_send_delay</literal> (<type>integer</type>)</term>
+ <listitem>
+ <para>
+ By default, the publisher sends changes as soon as possible. This
+ parameter allows the user to delay changes by given time period. If
+ the value is specified without units, it is taken as milliseconds.
+ The default is zero (no delay). See <xref linkend="config-setting-names-values"/>
+ for details on the available valid time units.
+ </para>
+ <para>
+ The delay is effective only when the initial table synchronization
+ has been finished. However, there is a possibility that the table
+ status written in <link linkend="catalog-pg-subscription-rel"><structname>pg_subscription_rel</structname></link>
+ will be delayed in getting to "ready" state, and also two-phase
+ (if specified) will be delayed in getting to "enabled".
+ </para>
+ <para>
+ The delay does not take into account the overhead of time spent
+ transferring the transaction. Therefore, the arrival time at the
+ subscriber may be delayed more than the specified
+ <literal>min_send_delay</literal> time.
+ </para>
+ <warning>
+ <para>
+ Delaying the replication means there is a much longer time between
+ making a change on the publisher, and that change being committed
+ on the subscriber. This can impact the performance of synchronous
+ replication. See <xref linkend="guc-synchronous-commit"/>
+ parameter.
+ </para>
+ </warning>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
</listitem>
</varlistentry>
@@ -420,6 +456,11 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
published with different column lists are not supported.
</para>
+ <para>
+ A non-zero <literal>min_send_delay</literal> parameter is not allowed when
+ streaming in parallel mode.
+ </para>
+
<para>
We allow non-existent publications to be specified so that users can add
those later. This means
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index a56ae311c3..63a10b06d1 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -66,6 +66,7 @@ GetSubscription(Oid subid, bool missing_ok)
sub->skiplsn = subform->subskiplsn;
sub->name = pstrdup(NameStr(subform->subname));
sub->owner = subform->subowner;
+ sub->minsenddelay = subform->subminsenddelay;
sub->enabled = subform->subenabled;
sub->binary = subform->subbinary;
sub->stream = subform->substream;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 34ca0e739f..54a705d71b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1314,9 +1314,10 @@ REVOKE ALL ON pg_replication_origin_status FROM public;
-- All columns of pg_subscription except subconninfo are publicly readable.
REVOKE ALL ON pg_subscription FROM public;
-GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
- subbinary, substream, subtwophasestate, subdisableonerr,
- subslotname, subsynccommit, subpublications, suborigin)
+GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subminsenddelay,
+ subenabled, subbinary, substream, subtwophasestate,
+ subdisableonerr, subslotname, subsynccommit, subpublications,
+ suborigin)
ON pg_subscription TO public;
CREATE VIEW pg_stat_subscription_stats AS
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 464db6d247..4a8cd47171 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -66,6 +66,7 @@
#define SUBOPT_DISABLE_ON_ERR 0x00000400
#define SUBOPT_LSN 0x00000800
#define SUBOPT_ORIGIN 0x00001000
+#define SUBOPT_MIN_SEND_DELAY 0x00002000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -90,6 +91,7 @@ typedef struct SubOpts
bool disableonerr;
char *origin;
XLogRecPtr lsn;
+ int32 min_send_delay;
} SubOpts;
static List *fetch_table_list(WalReceiverConn *wrconn, List *publications);
@@ -100,7 +102,7 @@ static void check_publications_origin(WalReceiverConn *wrconn,
static void check_duplicates_in_publist(List *publist, Datum *datums);
static List *merge_publications(List *oldpublist, List *newpublist, bool addpub, const char *subname);
static void ReportSlotConnectionError(List *rstates, Oid subid, char *slotname, char *err);
-
+static int32 defGetMinSendDelay(DefElem *def);
/*
* Common option parsing function for CREATE and ALTER SUBSCRIPTION commands.
@@ -146,6 +148,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->disableonerr = false;
if (IsSet(supported_opts, SUBOPT_ORIGIN))
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY))
+ opts->min_send_delay = 0;
/* Parse options */
foreach(lc, stmt_options)
@@ -324,6 +328,15 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->specified_opts |= SUBOPT_LSN;
opts->lsn = lsn;
}
+ else if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ if (IsSet(opts->specified_opts, SUBOPT_MIN_SEND_DELAY))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_MIN_SEND_DELAY;
+ opts->min_send_delay = defGetMinSendDelay(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -404,6 +417,29 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
"slot_name = NONE", "create_slot = false")));
}
}
+
+ /*
+ * The combination of parallel streaming mode and min_send_delay is not
+ * allowed. This is because in parallel streaming mode, the walsender
+ * starts sending the transaction stream without knowing the prepare/commit
+ * time of the transaction. Always waiting for the full 'min_send_delay'
+ * time to send may introduce unnecessary delay.
+ *
+ * The other possibility was to wait sending COMMIT record of the parallel
+ * apply transaction but that would cause issues related to resource bloat
+ * and locks being held for a long time.
+ */
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ opts->min_send_delay > 0 &&
+ opts->streaming == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ /*
+ * translator: the first %s is a string of the form "parameter > 0"
+ * and the second one is "option = value".
+ */
+ errmsg("%s and %s are mutually exclusive options",
+ "min_send_delay > 0", "streaming = parallel"));
}
/*
@@ -560,7 +596,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_SLOT_NAME | SUBOPT_COPY_DATA |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_TWOPHASE_COMMIT |
- SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN);
+ SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN |
+ SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -628,6 +665,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
values[Anum_pg_subscription_subname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->subname));
values[Anum_pg_subscription_subowner - 1] = ObjectIdGetDatum(owner);
+ values[Anum_pg_subscription_subminsenddelay - 1] = Int32GetDatum(opts.min_send_delay);
values[Anum_pg_subscription_subenabled - 1] = BoolGetDatum(opts.enabled);
values[Anum_pg_subscription_subbinary - 1] = BoolGetDatum(opts.binary);
values[Anum_pg_subscription_substream - 1] = CharGetDatum(opts.streaming);
@@ -1054,7 +1092,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
supported_opts = (SUBOPT_SLOT_NAME |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_DISABLE_ON_ERR |
- SUBOPT_ORIGIN);
+ SUBOPT_ORIGIN | SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options,
supported_opts, &opts);
@@ -1098,6 +1136,19 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
if (IsSet(opts.specified_opts, SUBOPT_STREAMING))
{
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.streaming == LOGICALREP_STREAM_PARALLEL &&
+ !IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY) &&
+ sub->minsenddelay > 0)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set parallel streaming mode for subscription with %s",
+ "min_send_delay"));
+
values[Anum_pg_subscription_substream - 1] =
CharGetDatum(opts.streaming);
replaces[Anum_pg_subscription_substream - 1] = true;
@@ -1111,6 +1162,26 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
= true;
}
+ if (IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY))
+ {
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.min_send_delay > 0 &&
+ !IsSet(opts.specified_opts, SUBOPT_STREAMING) &&
+ sub->stream == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set %s for subscription in parallel streaming mode",
+ "min_send_delay"));
+
+ values[Anum_pg_subscription_subminsenddelay - 1] =
+ Int32GetDatum(opts.min_send_delay);
+ replaces[Anum_pg_subscription_subminsenddelay - 1] = true;
+ }
+
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
{
values[Anum_pg_subscription_suborigin - 1] =
@@ -2195,3 +2266,45 @@ defGetStreamingMode(DefElem *def)
def->defname)));
return LOGICALREP_STREAM_OFF; /* keep compiler quiet */
}
+
+/*
+ * Extract the min_send_delay value from a DefElem. This is very similar to
+ * parse_and_validate_value() for integer values, because min_send_delay
+ * accepts the same parameter format as recovery_min_apply_delay.
+ */
+static int32
+defGetMinSendDelay(DefElem *def)
+{
+ char *input_string;
+ int result;
+ const char *hintmsg;
+
+ input_string = defGetString(def);
+
+ /*
+ * Parse given string as parameter which has millisecond unit
+ */
+ if (!parse_int(input_string, &result, GUC_UNIT_MS, &hintmsg))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid value for parameter \"%s\": \"%s\"",
+ "min_send_delay", input_string),
+ hintmsg ? errhint("%s", _(hintmsg)) : 0));
+
+ /*
+ * Check both the lower boundary for the valid min_send_delay range and
+ * the upper boundary as the safeguard for some platforms where INT_MAX is
+ * wider than int32 respectively. Although parse_int() has confirmed that
+ * the result is less than or equal to INT_MAX, the value will be stored
+ * in a catalog column of int32.
+ */
+ if (result < 0 || result > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%d ms is outside the valid range for parameter \"%s\" (%d .. %d)",
+ result,
+ "min_send_delay",
+ 0, PG_INT32_MAX)));
+
+ return result;
+}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 560ec974fa..89a72c1abe 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -443,6 +443,11 @@ libpqrcv_startstreaming(WalReceiverConn *conn,
PQserverVersion(conn->streamConn) >= 140000)
appendStringInfoString(&cmd, ", binary 'true'");
+ if (options->proto.logical.min_send_delay > 0 &&
+ PQserverVersion(conn->streamConn) >= 160000)
+ appendStringInfo(&cmd, ", min_send_delay '%d'",
+ options->proto.logical.min_send_delay);
+
appendStringInfoChar(&cmd, ')');
}
else
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index a53e23c679..2674444894 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -679,6 +679,15 @@ DecodeCommit(LogicalDecodingContext *ctx, XLogRecordBuffer *buf,
}
else
{
+ /*
+ * Delay sending the changes if required. For streaming transactions,
+ * this means a delay in sending the last stream but that is OK because
+ * on the downstream the changes will be applied only after receiving
+ * the last stream.
+ */
+ if (ctx->min_send_delay > 0 && ctx->delay_send)
+ ctx->delay_send(ctx, commit_time);
+
ReorderBufferCommit(ctx->reorder, xid, buf->origptr, buf->endptr,
commit_time, origin_id, origin_lsn);
}
@@ -763,6 +772,15 @@ DecodePrepare(LogicalDecodingContext *ctx, XLogRecordBuffer *buf,
buf->origptr, buf->endptr);
}
+ /*
+ * Delay sending the changes if required. For streaming transactions,
+ * this means a delay in sending the last stream but that is OK because
+ * on the downstream the changes will be applied only after receiving
+ * the last stream.
+ */
+ if (ctx->min_send_delay > 0 && ctx->delay_send)
+ ctx->delay_send(ctx, prepare_time);
+
/* replay actions of all transaction + subtransactions in order */
ReorderBufferPrepare(ctx->reorder, xid, parsed->twophase_gid);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c3ec97a0a6..e4dd822cdc 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -156,7 +156,8 @@ StartupDecodingContext(List *output_plugin_options,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send)
{
ReplicationSlot *slot;
MemoryContext context,
@@ -293,6 +294,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->prepare_write = prepare_write;
ctx->write = do_write;
ctx->update_progress = update_progress;
+ ctx->delay_send = delay_send;
ctx->output_plugin_options = output_plugin_options;
@@ -316,7 +318,7 @@ StartupDecodingContext(List *output_plugin_options,
* marking WAL reserved beforehand. In that scenario, it's up to the
* caller to guarantee that WAL remains available.
* xl_routine -- XLogReaderRoutine for underlying XLogReader
- * prepare_write, do_write, update_progress --
+ * prepare_write, do_write, update_progress, delay_send --
* callbacks that perform the use-case dependent, actual, work.
*
* Needs to be called while in a memory context that's at least as long lived
@@ -334,7 +336,8 @@ CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send)
{
TransactionId xmin_horizon = InvalidTransactionId;
ReplicationSlot *slot;
@@ -435,7 +438,7 @@ CreateInitDecodingContext(const char *plugin,
ctx = StartupDecodingContext(NIL, restart_lsn, xmin_horizon,
need_full_snapshot, false,
xl_routine, prepare_write, do_write,
- update_progress);
+ update_progress, delay_send);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
@@ -475,7 +478,7 @@ CreateInitDecodingContext(const char *plugin,
* xl_routine
* XLogReaderRoutine used by underlying xlogreader
*
- * prepare_write, do_write, update_progress
+ * prepare_write, do_write, update_progress, delay_send
* callbacks that have to be filled to perform the use-case dependent,
* actual work.
*
@@ -493,7 +496,8 @@ CreateDecodingContext(XLogRecPtr start_lsn,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send)
{
LogicalDecodingContext *ctx;
ReplicationSlot *slot;
@@ -547,7 +551,7 @@ CreateDecodingContext(XLogRecPtr start_lsn,
ctx = StartupDecodingContext(output_plugin_options,
start_lsn, InvalidTransactionId, false,
fast_forward, xl_routine, prepare_write,
- do_write, update_progress);
+ do_write, update_progress, delay_send);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c
index fa1b641a2b..960025197f 100644
--- a/src/backend/replication/logical/logicalfuncs.c
+++ b/src/backend/replication/logical/logicalfuncs.c
@@ -212,7 +212,7 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
LogicalOutputPrepareWrite,
- LogicalOutputWrite, NULL);
+ LogicalOutputWrite, NULL, NULL);
/*
* After the sanity checks in CreateDecodingContext, make sure the
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index cfb2ab6248..e68902ae34 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -3898,7 +3898,8 @@ maybe_reread_subscription(void)
newsub->stream != MySubscription->stream ||
strcmp(newsub->origin, MySubscription->origin) != 0 ||
newsub->owner != MySubscription->owner ||
- !equal(newsub->publications, MySubscription->publications))
+ !equal(newsub->publications, MySubscription->publications) ||
+ newsub->minsenddelay != MySubscription->minsenddelay)
{
if (am_parallel_apply_worker())
ereport(LOG,
@@ -4617,9 +4618,18 @@ ApplyWorkerMain(Datum main_arg)
options.proto.logical.twophase = false;
options.proto.logical.origin = pstrdup(MySubscription->origin);
+ options.proto.logical.min_send_delay = 0;
if (!am_tablesync_worker())
{
+ /*
+ * Time-delayed logical replication does not support tablesync
+ * workers, so only the leader apply worker can request walsenders to
+ * delay on the publisher side.
+ */
+ if (server_version >= 160000 && MySubscription->minsenddelay > 0)
+ options.proto.logical.min_send_delay = MySubscription->minsenddelay;
+
/*
* Even when the two_phase mode is requested by the user, it remains
* as the tri-state PENDING until all tablesyncs have reached READY
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 98377c094b..b7271d0f8a 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -285,6 +285,7 @@ parse_output_parameters(List *options, PGOutputData *data)
bool streaming_given = false;
bool two_phase_option_given = false;
bool origin_option_given = false;
+ bool min_send_delay_option_given = false;
data->binary = false;
data->streaming = LOGICALREP_STREAM_OFF;
@@ -396,6 +397,32 @@ parse_output_parameters(List *options, PGOutputData *data)
errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized origin value: \"%s\"", data->origin));
}
+ else if (strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ unsigned long parsed;
+ char *endptr;
+
+ if (min_send_delay_option_given)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("conflicting or redundant options"));
+ min_send_delay_option_given = true;
+
+ errno = 0;
+ parsed = strtoul(strVal(defel->arg), &endptr, 10);
+ if (errno != 0 || *endptr != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid min_send_delay")));
+
+ if (parsed > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("min_send_delay \"%s\" out of range",
+ strVal(defel->arg))));
+
+ data->min_send_delay = (int32) parsed;
+ }
else
elog(ERROR, "unrecognized pgoutput option: %s", defel->defname);
}
@@ -501,6 +528,15 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
else
ctx->twophase_opt_given = true;
+ if (data->min_send_delay &&
+ data->protocol_version < LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("requested proto_version=%d does not support delay sending data, need %d or higher",
+ data->protocol_version, LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM)));
+ else
+ ctx->min_send_delay = data->min_send_delay;
+
/* Init publication state. */
data->publications = NIL;
publications_valid = false;
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 2f3c964824..522f7600a1 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -148,7 +148,7 @@ create_logical_replication_slot(char *name, char *plugin,
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* If caller needs us to determine the decoding start point, do so now.
@@ -481,7 +481,7 @@ pg_logical_replication_slot_advance(XLogRecPtr moveto)
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* Start reading at the slot's restart_lsn, which we know to point to
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 75e8363e24..9537fba7df 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -252,6 +252,7 @@ static void WalSndPrepareWrite(LogicalDecodingContext *ctx, XLogRecPtr lsn, Tran
static void WalSndWriteData(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid, bool last_write);
static void WalSndUpdateProgress(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid,
bool skipped_xact);
+static void WalSndDelay(LogicalDecodingContext *ctx, TimestampTz delay_start);
static XLogRecPtr WalSndWaitForWal(XLogRecPtr loc);
static void LagTrackerWrite(XLogRecPtr lsn, TimestampTz local_flush_time);
static TimeOffset LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now);
@@ -1126,7 +1127,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
/*
* Signal that we don't need the timeout mechanism. We're just
@@ -1285,7 +1286,7 @@ StartLogicalReplication(StartReplicationCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
xlogreader = logical_decoding_ctx->reader;
WalSndSetState(WALSNDSTATE_CATCHUP);
@@ -3849,3 +3850,76 @@ LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now)
Assert(time != 0);
return now - time;
}
+
+/*
+ * LogicalDecodingContext 'delay' callback.
+ *
+ * Wait long enough to make sure a transaction is applied at least that
+ * period behind the publisher.
+ */
+static void
+WalSndDelay(LogicalDecodingContext *ctx, TimestampTz delay_start)
+{
+ /* Wait till delayUntil by the latch mechanism */
+ while (true)
+ {
+ TimestampTz delayUntil;
+ long remaining_wait_time_ms;
+ long timeout_sleeptime_ms;
+
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* This might change wal_sender_timeout */
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+
+ /* Check for input from the client */
+ ProcessRepliesIfAny();
+
+ /* die if timeout was reached */
+ WalSndCheckTimeOut();
+
+ /* Send keepalive if the time has come */
+ WalSndKeepaliveIfNecessary();
+
+ /* Try to flush pending output to the client */
+ if (pq_flush_if_writable() != 0)
+ WalSndShutdown();
+
+ /*
+ * If we've requested to shut down, exit the process.
+ *
+ * Note that WalSndDone() cannot be used here because the delaying
+ * changes will be sent in the function.
+ */
+ if (got_STOPPING)
+ WalSndShutdown();
+
+ delayUntil = TimestampTzPlusMilliseconds(delay_start, ctx->min_send_delay);
+ remaining_wait_time_ms = TimestampDifferenceMilliseconds(GetCurrentTimestamp(), delayUntil);
+
+ /*
+ * Exit without arming the latch if it's already past time to send
+ * this transaction.
+ */
+ if (remaining_wait_time_ms <= 0)
+ break;
+
+ /* Sleep until appropriate time. */
+ timeout_sleeptime_ms = WalSndComputeSleeptime(GetCurrentTimestamp());
+
+ elog(DEBUG2, "time-delayed replication for txid %u, delay_time = %d ms, remaining wait time: %ld ms",
+ ctx->write_xid, (int) ctx->min_send_delay,
+ remaining_wait_time_ms);
+
+ /* Sleep until we get reply from worker or we time out */
+ WalSndWait(WL_SOCKET_READABLE,
+ Min(timeout_sleeptime_ms, remaining_wait_time_ms),
+ WAIT_EVENT_WALSENDER_SEND_DELAY);
+ }
+}
diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c
index cb99cc6339..76c19fe11d 100644
--- a/src/backend/utils/activity/wait_event.c
+++ b/src/backend/utils/activity/wait_event.c
@@ -515,6 +515,9 @@ pgstat_get_wait_timeout(WaitEventTimeout w)
case WAIT_EVENT_VACUUM_TRUNCATE:
event_name = "VacuumTruncate";
break;
+ case WAIT_EVENT_WALSENDER_SEND_DELAY:
+ event_name = "WalSenderSendDelay";
+ break;
/* no default case, so that compiler will warn */
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 527c7651ab..bd95747840 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4494,6 +4494,7 @@ getSubscriptions(Archive *fout)
int i_subsynccommit;
int i_subpublications;
int i_subbinary;
+ int i_subminsenddelay;
int i,
ntups;
@@ -4546,9 +4547,13 @@ getSubscriptions(Archive *fout)
LOGICALREP_TWOPHASE_STATE_DISABLED);
if (fout->remoteVersion >= 160000)
- appendPQExpBufferStr(query, " s.suborigin\n");
+ appendPQExpBufferStr(query,
+ " s.suborigin,\n"
+ " s.subminsenddelay\n");
else
- appendPQExpBuffer(query, " '%s' AS suborigin\n", LOGICALREP_ORIGIN_ANY);
+ appendPQExpBuffer(query, " '%s' AS suborigin,\n"
+ " 0 AS subminsenddelay\n",
+ LOGICALREP_ORIGIN_ANY);
appendPQExpBufferStr(query,
"FROM pg_subscription s\n"
@@ -4576,6 +4581,7 @@ getSubscriptions(Archive *fout)
i_subtwophasestate = PQfnumber(res, "subtwophasestate");
i_subdisableonerr = PQfnumber(res, "subdisableonerr");
i_suborigin = PQfnumber(res, "suborigin");
+ i_subminsenddelay = PQfnumber(res, "subminsenddelay");
subinfo = pg_malloc(ntups * sizeof(SubscriptionInfo));
@@ -4606,6 +4612,8 @@ getSubscriptions(Archive *fout)
subinfo[i].subdisableonerr =
pg_strdup(PQgetvalue(res, i, i_subdisableonerr));
subinfo[i].suborigin = pg_strdup(PQgetvalue(res, i, i_suborigin));
+ subinfo[i].subminsenddelay =
+ atoi(PQgetvalue(res, i, i_subminsenddelay));
/* Decide whether we want to dump it */
selectDumpableObject(&(subinfo[i].dobj), fout);
@@ -4687,6 +4695,9 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
+ if (subinfo->subminsenddelay > 0)
+ appendPQExpBuffer(query, ", min_send_delay = '%d ms'", subinfo->subminsenddelay);
+
appendPQExpBufferStr(query, ");\n");
if (subinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e7cbd8d7ed..24e0f6737f 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -661,6 +661,7 @@ typedef struct _SubscriptionInfo
char *subdisableonerr;
char *suborigin;
char *subsynccommit;
+ int subminsenddelay;
char *subpublications;
} SubscriptionInfo;
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index c8a0bb7b3a..c7d303a168 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6472,7 +6472,7 @@ describeSubscriptions(const char *pattern, bool verbose)
PGresult *res;
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
- false, false, false, false, false, false, false, false};
+ false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6527,10 +6527,13 @@ describeSubscriptions(const char *pattern, bool verbose)
gettext_noop("Two-phase commit"),
gettext_noop("Disable on error"));
+ /* Origin and min_send_delay are only supported in v16 and higher */
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
- ", suborigin AS \"%s\"\n",
- gettext_noop("Origin"));
+ ", suborigin AS \"%s\"\n"
+ ", subminsenddelay AS \"%s\"\n",
+ gettext_noop("Origin"),
+ gettext_noop("Min send delay"));
appendPQExpBuffer(&buf,
", subsynccommit AS \"%s\"\n"
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5e1882eaea..6643db6f55 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1925,7 +1925,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("(", "PUBLICATION");
/* ALTER SUBSCRIPTION <name> SET ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SET", "("))
- COMPLETE_WITH("binary", "disable_on_error", "origin", "slot_name",
+ COMPLETE_WITH("binary", "disable_on_error", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit");
/* ALTER SUBSCRIPTION <name> SKIP ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SKIP", "("))
@@ -3268,7 +3268,7 @@ psql_completion(const char *text, int start, int end)
/* Complete "CREATE SUBSCRIPTION <name> ... WITH ( <opt>" */
else if (HeadMatches("CREATE", "SUBSCRIPTION") && TailMatches("WITH", "("))
COMPLETE_WITH("binary", "connect", "copy_data", "create_slot",
- "disable_on_error", "enabled", "origin", "slot_name",
+ "disable_on_error", "enabled", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit", "two_phase");
/* CREATE TRIGGER --- is allowed inside CREATE SCHEMA, so use TailMatches */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index b0f2a1705d..69ae4314b4 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -74,6 +74,8 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
Oid subowner BKI_LOOKUP(pg_authid); /* Owner of the subscription */
+ int32 subminsenddelay; /* Replication send delay (ms) */
+
bool subenabled; /* True if the subscription is enabled (the
* worker should be running) */
@@ -122,6 +124,7 @@ typedef struct Subscription
* skipped */
char *name; /* Name of the subscription */
Oid owner; /* Oid of the subscription owner */
+ int32 minsenddelay; /* Replication send delay (ms) */
bool enabled; /* Indicates if the subscription is enabled */
bool binary; /* Indicates if the subscription wants data in
* binary format */
diff --git a/src/include/replication/logical.h b/src/include/replication/logical.h
index 5f49554ea0..ae6d873a94 100644
--- a/src/include/replication/logical.h
+++ b/src/include/replication/logical.h
@@ -30,6 +30,10 @@ typedef void (*LogicalOutputPluginWriterUpdateProgress) (struct LogicalDecodingC
bool skipped_xact
);
+typedef void (*LogicalOutputPluginWriterDelay) (struct LogicalDecodingContext *lr,
+ TimestampTz start_time
+);
+
typedef struct LogicalDecodingContext
{
/* memory context this is all allocated in */
@@ -64,6 +68,7 @@ typedef struct LogicalDecodingContext
LogicalOutputPluginWriterPrepareWrite prepare_write;
LogicalOutputPluginWriterWrite write;
LogicalOutputPluginWriterUpdateProgress update_progress;
+ LogicalOutputPluginWriterDelay delay_send;
/*
* Output buffer.
@@ -100,6 +105,12 @@ typedef struct LogicalDecodingContext
*/
bool twophase_opt_given;
+ /*
+ * The minimum delay, in milliseconds, by the publisher before sending
+ * COMMIT/PREPARE record
+ */
+ int32 min_send_delay;
+
/*
* State for writing output.
*/
@@ -121,14 +132,16 @@ extern LogicalDecodingContext *CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send);
extern LogicalDecodingContext *CreateDecodingContext(XLogRecPtr start_lsn,
List *output_plugin_options,
bool fast_forward,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send);
extern void DecodingContextFindStartpoint(LogicalDecodingContext *ctx);
extern bool DecodingContextReady(LogicalDecodingContext *ctx);
extern void FreeDecodingContext(LogicalDecodingContext *ctx);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index 0ea2df5088..46faadbd7a 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -36,13 +36,17 @@
* LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM is the minimum protocol version
* where we support applying large streaming transactions in parallel.
* Introduced in PG16.
+ *
+ * LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM is the minimum protocol version
+ * with support for delaying to send transactions. Introduced in PG16.
*/
#define LOGICALREP_PROTO_MIN_VERSION_NUM 1
#define LOGICALREP_PROTO_VERSION_NUM 1
#define LOGICALREP_PROTO_STREAM_VERSION_NUM 2
#define LOGICALREP_PROTO_TWOPHASE_VERSION_NUM 3
#define LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM 4
-#define LOGICALREP_PROTO_MAX_VERSION_NUM LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM
+#define LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM 4
+#define LOGICALREP_PROTO_MAX_VERSION_NUM LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM
/*
* Logical message types
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index b4a8015403..d2fde09e00 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -30,6 +30,7 @@ typedef struct PGOutputData
bool messages;
bool two_phase;
char *origin;
+ int32 min_send_delay;
} PGOutputData;
#endif /* PGOUTPUT_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index decffe352d..c20969aed7 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -187,6 +187,7 @@ typedef struct
* prepare time */
char *origin; /* Only publish data originating from the
* specified origin */
+ int32 min_send_delay; /* The minimum send delay */
} logical;
} proto;
} WalRcvStreamOptions;
diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h
index 9ab23e1c4a..cc3a234eba 100644
--- a/src/include/utils/wait_event.h
+++ b/src/include/utils/wait_event.h
@@ -150,7 +150,8 @@ typedef enum
WAIT_EVENT_REGISTER_SYNC_REQUEST,
WAIT_EVENT_SPIN_DELAY,
WAIT_EVENT_VACUUM_DELAY,
- WAIT_EVENT_VACUUM_TRUNCATE
+ WAIT_EVENT_VACUUM_TRUNCATE,
+ WAIT_EVENT_WALSENDER_SEND_DELAY
} WaitEventTimeout;
/* ----------
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 3f99b14394..2027316233 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -114,18 +114,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -143,10 +143,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -163,10 +163,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/12345
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/12345
(1 row)
-- ok - with lsn = NONE
@@ -175,10 +175,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
BEGIN;
@@ -210,10 +210,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
ERROR: invalid value for parameter "synchronous_commit": "foobar"
HINT: Available values: local, remote_write, remote_apply, on, off.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | local | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | local | dbname=regress_doesnotexist2 | 0/0
(1 row)
-- rename back to keep the rest simple
@@ -247,19 +247,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -271,27 +271,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication already exists
@@ -306,10 +306,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication used more than once
@@ -324,10 +324,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -363,10 +363,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
--fail - alter of two_phase option not supported.
@@ -375,10 +375,10 @@ ERROR: unrecognized subscription parameter: "two_phase"
-- but can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -388,10 +388,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -404,20 +404,61 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | 0 | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+ERROR: invalid value for parameter "min_send_delay": "foo"
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+ERROR: -1 ms is outside the valid range for parameter "min_send_delay" (0 .. 2147483647)
+-- fail - specifying streaming = parallel with time-delayed replication is not
+-- supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+ERROR: min_send_delay > 0 and streaming = parallel are mutually exclusive options
+-- success -- min_send_delay value without units is taken as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 123 | off | dbname=regress_doesnotexit | 0/0
+(1 row)
+
+-- success -- min_send_delay value with units other than ms is converted to ms
+-- and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 86400000 | off | dbname=regress_doesnotexit | 0/0
(1 row)
+-- fail - alter subscription with streaming = parallel should fail when
+-- time-delayed replication is set
+ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
+ERROR: cannot set parallel streaming mode for subscription with min_send_delay
+-- fail - alter subscription with min_send_delay should fail when
+-- streaming = parallel is set
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 0, streaming = parallel);
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 123);
+ERROR: cannot set min_send_delay for subscription in parallel streaming mode
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 7281f5fee2..46bf4a27d9 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -286,6 +286,35 @@ ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+
+-- fail - specifying streaming = parallel with time-delayed replication is not
+-- supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+
+-- success -- min_send_delay value without units is taken as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+\dRs+
+
+-- success -- min_send_delay value with units other than ms is converted to ms
+-- and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+
+-- fail - alter subscription with streaming = parallel should fail when
+-- time-delayed replication is set
+ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
+
+-- fail - alter subscription with min_send_delay should fail when
+-- streaming = parallel is set
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 0, streaming = parallel);
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 123);
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
RESET SESSION AUTHORIZATION;
DROP ROLE regress_subscription_user;
DROP ROLE regress_subscription_user2;
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 91aa068c95..063a98fde9 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -515,6 +515,33 @@ $node_publisher->poll_query_until('postgres',
or die
"Timed out while waiting for apply to restart after renaming SUBSCRIPTION";
+# Test time-delayed logical replication
+#
+# If the subscription sets min_send_delay parameter, the walsender will delay
+# the transaction send for min_send_delay milliseconds. We verify this by
+# looking at the time difference between a) when tuples are inserted on the
+# publisher, and b) when those changes are replicated on the subscriber. Even
+# on slow machines, this strategy will give predictable behavior.
+
+# Set min_send_delay parameter to 3 seconds
+my $delay = 3;
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub_renamed SET (min_send_delay = '${delay}s')");
+
+# Before doing the insertion, get the current timestamp that will be
+# used as a comparison base.
+my $publisher_insert_time = time();
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_ins VALUES (generate_series(1101, 1120))");
+
+# The publisher waits for the replication to complete
+$node_publisher->wait_for_catchup('tap_sub_renamed');
+
+# This test is successful only if at least the configured delay has elapsed.
+ok( time() - $publisher_insert_time >= $delay,
+ "subscriber applies WAL only after replication delay for non-streaming transaction"
+);
+
# check all the cleanup
$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub_renamed");
--
2.27.0
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-21 08:03 Hayato Kuroda (Fujitsu) <[email protected]>
parent: Amit Kapila <[email protected]>
0 siblings, 0 replies; 17+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2023-02-21 08:03 UTC (permalink / raw)
To: 'Amit Kapila' <[email protected]>; Peter Smith <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Dear Amit,
Thank you for commenting!
> > 8.
> > + <para>
> > + The delay is effective only when the initial table synchronization
> > + has been finished and the publisher decides to send a particular
> > + transaction downstream. The delay does not take into account the
> > + overhead of time spent in transferring the transaction, which
> means
> > + that the arrival time at the subscriber may be delayed more than
> the
> > + given time.
> > + </para>
> >
> > I'm not sure about this mention about only "effective only when the
> > initial table synchronization has been finished"... Now that the delay
> > is pub-side I don't know that it is true anymore.
> >
>
> This will still be true because we don't wait during the initial copy
> (sync). The delay happens only when the replication starts.
Maybe this depends on the definition of initial copy and sync.
I checked and added descriptions in [1].
> > 11.
> > + errno = 0;
> > + parsed = strtoul(strVal(defel->arg), &endptr, 10);
> > + if (errno != 0 || *endptr != '\0')
> > + ereport(ERROR,
> > + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> > + errmsg("invalid min_send_delay")));
> > +
> > + if (parsed > PG_INT32_MAX)
> > + ereport(ERROR,
> > + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> > + errmsg("min_send_delay \"%s\" out of range",
> > + strVal(defel->arg))));
> >
> > Should the validation be also checking/asserting no negative numbers,
> > or actually should the min_send_delay be defined as a uint32 in the
> > first place?
> >
>
> I don't see the need to change the datatype of min_send_delay as
> compared to what we have min_apply_delay.
I think it is OK to change "long" to "unsinged long", because
We use strtoul() for reading and should reject the minus value.
Of course we can modify them, but I want to keep the consistency with proto_version part.
[1]: https://www.postgresql.org/message-id/[email protected]....
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-22 11:55 Amit Kapila <[email protected]>
parent: Hayato Kuroda (Fujitsu) <[email protected]>
0 siblings, 1 reply; 17+ messages in thread
From: Amit Kapila @ 2023-02-22 11:55 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Peter Smith <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
On Tue, Feb 21, 2023 at 1:28 PM Hayato Kuroda (Fujitsu)
<[email protected]> wrote:
>
> > doc/src/sgml/catalogs.sgml
> >
> > 4.
> > + <row>
> > + <entry role="catalog_table_entry"><para role="column_definition">
> > + <structfield>subminsenddelay</structfield> <type>int4</type>
> > + </para>
> > + <para>
> > + The minimum delay, in milliseconds, by the publisher to send changes
> > + </para></entry>
> > + </row>
> >
> > "by the publisher to send changes" --> "by the publisher before sending changes"
>
> As Amit said[1], there is a possibility to delay after sending delay. So I changed to
> "before sending COMMIT record". How do you think?
>
I think it would be better to say: "The minimum delay, in
milliseconds, by the publisher before sending all the changes". If you
agree then similar change is required in below comment as well:
+ /*
+ * The minimum delay, in milliseconds, by the publisher before sending
+ * COMMIT/PREPARE record.
+ */
+ int32 min_send_delay;
+
>
> > src/backend/replication/pgoutput/pgoutput.c
> >
> > 11.
> > + errno = 0;
> > + parsed = strtoul(strVal(defel->arg), &endptr, 10);
> > + if (errno != 0 || *endptr != '\0')
> > + ereport(ERROR,
> > + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> > + errmsg("invalid min_send_delay")));
> > +
> > + if (parsed > PG_INT32_MAX)
> > + ereport(ERROR,
> > + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> > + errmsg("min_send_delay \"%s\" out of range",
> > + strVal(defel->arg))));
> >
> > Should the validation be also checking/asserting no negative numbers,
> > or actually should the min_send_delay be defined as a uint32 in the
> > first place?
>
> I think you are right because min_apply_delay does not have related code.
> we must consider additional possibility that user may send START_REPLICATION
> by hand and it has minus value.
> Fixed.
>
Your reasoning for adding the additional check seems good to me but I
don't see it in the patch. The check as I see is as below:
+ if (delay_val > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("min_send_delay \"%s\" out of range",
+ strVal(defel->arg))));
Am, I missing something, and the new check is at some other place?
+ has been finished. However, there is a possibility that the table
+ status written in <link
linkend="catalog-pg-subscription-rel"><structname>pg_subscription_rel</structname></link>
+ will be delayed in getting to "ready" state, and also two-phase
+ (if specified) will be delayed in getting to "enabled".
+ </para>
There appears to be a special value <0x00> after "ready". I think that
is added by mistake or probably you have used some editor which has
added this value. Can we slightly reword this to: "However, there is a
possibility that the table status updated in <link
linkend="catalog-pg-subscription-rel"><structname>pg_subscription_rel</structname></link>
could be delayed in getting to the "ready" state, and also two-phase
(if specified) could be delayed in getting to "enabled"."?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-22 13:47 Hayato Kuroda (Fujitsu) <[email protected]>
parent: Amit Kapila <[email protected]>
0 siblings, 2 replies; 17+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2023-02-22 13:47 UTC (permalink / raw)
To: 'Amit Kapila' <[email protected]>; +Cc: Peter Smith <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Dear Amit,
Thank you for reviewing! PSA new version.
> I think it would be better to say: "The minimum delay, in
> milliseconds, by the publisher before sending all the changes". If you
> agree then similar change is required in below comment as well:
> + /*
> + * The minimum delay, in milliseconds, by the publisher before sending
> + * COMMIT/PREPARE record.
> + */
> + int32 min_send_delay;
OK, both of them were fixed.
> > > Should the validation be also checking/asserting no negative numbers,
> > > or actually should the min_send_delay be defined as a uint32 in the
> > > first place?
> >
> > I think you are right because min_apply_delay does not have related code.
> > we must consider additional possibility that user may send
> START_REPLICATION
> > by hand and it has minus value.
> > Fixed.
> >
>
> Your reasoning for adding the additional check seems good to me but I
> don't see it in the patch. The check as I see is as below:
> + if (delay_val > PG_INT32_MAX)
> + ereport(ERROR,
> + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> + errmsg("min_send_delay \"%s\" out of range",
> + strVal(defel->arg))));
>
> Am, I missing something, and the new check is at some other place?
For extracting value from the string, strtoul() is used.
This is an important point.
```
delay_val = strtoul(strVal(defel->arg), &endptr, 10);
```
If user specifies min_send_delay as '-1', the value is read as a bit string
'0xFFFFFFFFFFFFFFFF', and it is interpreted as PG_UINT64_MAX. After that such a
strange value is rejected by the part you copied. I have tested the case and it has
correctly rejected.
```
postgres=# START_REPLICATION SLOT "sub" LOGICAL 0/0 (min_send_delay '-1');
ERROR: min_send_delay "-1" out of range
CONTEXT: slot "sub", output plugin "pgoutput", in the startup callback
```
> + has been finished. However, there is a possibility that the table
> + status written in <link
> linkend="catalog-pg-subscription-rel"><structname>pg_subscription_rel</stru
> ctname></link>
> + will be delayed in getting to "ready" state, and also two-phase
> + (if specified) will be delayed in getting to "enabled".
> + </para>
>
> There appears to be a special value <0x00> after "ready". I think that
> is added by mistake or probably you have used some editor which has
> added this value. Can we slightly reword this to: "However, there is a
> possibility that the table status updated in <link
> linkend="catalog-pg-subscription-rel"><structname>pg_subscription_rel</stru
> ctname></link>
> could be delayed in getting to the "ready" state, and also two-phase
> (if specified) could be delayed in getting to "enabled"."?
Oh, my Visual Studio Code did not detect the strange character.
And reworded accordingly.
Additionally, I modified the commit message to describe more clearly the reason
why the do not allow combination of min_send_delay and streaming = parallel.
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
Attachments:
[application/octet-stream] v6-0001-Time-delayed-logical-replication-on-publisher-sid.patch (82.6K, ../../TYAPR01MB5866982AD09BD79D2536675EF5AA9@TYAPR01MB5866.jpnprd01.prod.outlook.com/2-v6-0001-Time-delayed-logical-replication-on-publisher-sid.patch)
download | inline diff:
From 6721d1747e055478d204ce479ba883659bfb5301 Mon Sep 17 00:00:00 2001
From: Hayato Kuroda <[email protected]>
Date: Thu, 16 Feb 2023 07:52:23 +0000
Subject: [PATCH v6] Time-delayed logical replication on publisher side
Similar to physical replication, a time-delayed copy of the data for
logical replication is useful for some scenarios (particularly to fix
errors that might cause data loss).
This patch implements a new subscription parameter called 'min_send_delay'.
If the subscription sets min_send_delay parameter, the apply worker
(via walrcv_startstreaming) passes the value to the publisher as an output plugin
option. The walsender will delay the transaction sending for given milliseconds.
The delay does not take into account the overhead of time spent in transferring
the transaction, which means that the arrival time at the subscriber may be
delayed more than the given time.
The delay occurs before we start to send the transaction on the publisher.
Regular and prepared transactions are covered. Streamed transactions are also
covered.
The combination of parallel streaming mode and min_send_delay is not allowed
1. This is because in parallel streaming mode, we start applying the transaction
stream as soon as the first change arrives without knowing the transaction's
prepare/commit time. Always waiting for the full 'min_send_delay' period might
include unnecessary delay.
2. Another reason is, for parallel streaming, the transaction will be opened
immediately by the parallel apply worker. So if walsender delayed to send the
final record of the transaction, the parallel worker must wait receiving with an
opened transaction. This would lead that locks acquired during the transaction
not getting released till min_send_delay elapsed.
Earlier versions were written by Euler Taveira, Takamichi Osumi, and Kuroda Hayato
Author: Kuroda Hayato, Takamichi Osumi
Reviewed-by: Amit Kapila, Peter Smith, Vignesh C, Shveta Malik,
Kyotaro Horiguchi, Shi Yu, Wang Wei, Dilip Kumar, Melih Mutlu,
Andres Freund
---
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/glossary.sgml | 15 ++
doc/src/sgml/logical-replication.sgml | 6 +
doc/src/sgml/monitoring.sgml | 5 +
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 43 +++-
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 7 +-
src/backend/commands/subscriptioncmds.c | 119 ++++++++++-
.../libpqwalreceiver/libpqwalreceiver.c | 5 +
src/backend/replication/logical/decode.c | 18 ++
src/backend/replication/logical/logical.c | 18 +-
.../replication/logical/logicalfuncs.c | 2 +-
src/backend/replication/logical/worker.c | 12 +-
src/backend/replication/pgoutput/pgoutput.c | 36 ++++
src/backend/replication/slotfuncs.c | 4 +-
src/backend/replication/walsender.c | 78 +++++++-
src/backend/utils/activity/wait_event.c | 3 +
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 9 +-
src/bin/psql/tab-complete.c | 4 +-
src/include/catalog/pg_subscription.h | 3 +
src/include/replication/logical.h | 17 +-
src/include/replication/logicalproto.h | 6 +-
src/include/replication/pgoutput.h | 1 +
src/include/replication/walreceiver.h | 1 +
src/include/utils/wait_event.h | 3 +-
src/test/regress/expected/subscription.out | 185 +++++++++++-------
src/test/regress/sql/subscription.sql | 29 +++
src/test/subscription/t/001_rep_changes.pl | 27 +++
31 files changed, 583 insertions(+), 105 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c1e4048054..ce6521fe6c 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -7892,6 +7892,16 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subminsenddelay</structfield> <type>int4</type>
+ </para>
+ <para>
+ The minimum delay, in milliseconds, by the publisher before sending all
+ the changes
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subenabled</structfield> <type>bool</type>
diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml
index 7c01a541fe..9ede9d05f6 100644
--- a/doc/src/sgml/glossary.sgml
+++ b/doc/src/sgml/glossary.sgml
@@ -1729,6 +1729,21 @@
</glossdef>
</glossentry>
+ <glossentry id="glossary-time-delayed-replication">
+ <glossterm>Time-delayed replication</glossterm>
+ <glossdef>
+ <para>
+ Replication setup that delays the application of changes by a specified
+ minimum time-delay period.
+ </para>
+ <para>
+ For more information, see
+ <xref linkend="guc-recovery-min-apply-delay"/> for physical replication
+ and <xref linkend="sql-createsubscription"/> for logical replication.
+ </para>
+ </glossdef>
+ </glossentry>
+
<glossentry id="glossary-toast">
<glossterm>TOAST</glossterm>
<glossdef>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 1bd5660c87..fe9e7f7b26 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -247,6 +247,12 @@
target table.
</para>
+ <para>
+ A subscription can delay the receipt of changes by specifying the
+ <literal>min_send_delay</literal> subscription parameter. See
+ <xref linkend="sql-createsubscription"/> for details.
+ </para>
+
<sect2 id="logical-replication-subscription-slot">
<title>Replication Slot Management</title>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b0b997f092..6158587644 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -2349,6 +2349,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry>Waiting to acquire an exclusive lock to truncate off any
empty pages at the end of a table vacuumed.</entry>
</row>
+ <row>
+ <entry><literal>WalSenderSendDelay</literal></entry>
+ <entry>Waiting while sending changes for time-delayed logical replication
+ in the WAL sender process.</entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 964fcbb8ff..3f238b958b 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -213,8 +213,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
are <literal>slot_name</literal>,
<literal>synchronous_commit</literal>,
<literal>binary</literal>, <literal>streaming</literal>,
- <literal>disable_on_error</literal>, and
- <literal>origin</literal>.
+ <literal>disable_on_error</literal>,
+ <literal>origin</literal>, and
+ <literal>min_send_delay</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 51c45f17c7..9d08740ba2 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -349,7 +349,43 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
- </variablelist></para>
+
+ <varlistentry>
+ <term><literal>min_send_delay</literal> (<type>integer</type>)</term>
+ <listitem>
+ <para>
+ By default, the publisher sends changes as soon as possible. This
+ parameter allows the user to delay changes by the given time period.
+ If the value is specified without units, it is taken as milliseconds.
+ The default is zero (no delay). See <xref linkend="config-setting-names-values"/>
+ for details on the available valid time units.
+ </para>
+ <para>
+ The delay is effective only when the initial table synchronization
+ has been finished. However, there is a possibility that the table
+ status updated in <link linkend="catalog-pg-subscription-rel"><structname>pg_subscription_rel</structname></link>
+ could be delayed in getting to the "ready" state, and also two-phase
+ (if specified) could be delayed in getting to "enabled".
+ </para>
+ <para>
+ The delay does not take into account the overhead of time spent
+ transferring the transaction. Therefore, the arrival time at the
+ subscriber may be delayed more than the specified
+ <literal>min_send_delay</literal> time.
+ </para>
+ <warning>
+ <para>
+ Delaying the replication means there is a much longer time between
+ making a change on the publisher, and that change being committed
+ on the subscriber. This can impact the performance of synchronous
+ replication. See <xref linkend="guc-synchronous-commit"/>
+ parameter.
+ </para>
+ </warning>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
</listitem>
</varlistentry>
@@ -420,6 +456,11 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
published with different column lists are not supported.
</para>
+ <para>
+ A non-zero <literal>min_send_delay</literal> parameter is not allowed when
+ streaming in parallel mode.
+ </para>
+
<para>
We allow non-existent publications to be specified so that users can add
those later. This means
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index a56ae311c3..63a10b06d1 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -66,6 +66,7 @@ GetSubscription(Oid subid, bool missing_ok)
sub->skiplsn = subform->subskiplsn;
sub->name = pstrdup(NameStr(subform->subname));
sub->owner = subform->subowner;
+ sub->minsenddelay = subform->subminsenddelay;
sub->enabled = subform->subenabled;
sub->binary = subform->subbinary;
sub->stream = subform->substream;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 34ca0e739f..54a705d71b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1314,9 +1314,10 @@ REVOKE ALL ON pg_replication_origin_status FROM public;
-- All columns of pg_subscription except subconninfo are publicly readable.
REVOKE ALL ON pg_subscription FROM public;
-GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
- subbinary, substream, subtwophasestate, subdisableonerr,
- subslotname, subsynccommit, subpublications, suborigin)
+GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subminsenddelay,
+ subenabled, subbinary, substream, subtwophasestate,
+ subdisableonerr, subslotname, subsynccommit, subpublications,
+ suborigin)
ON pg_subscription TO public;
CREATE VIEW pg_stat_subscription_stats AS
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 464db6d247..4a8cd47171 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -66,6 +66,7 @@
#define SUBOPT_DISABLE_ON_ERR 0x00000400
#define SUBOPT_LSN 0x00000800
#define SUBOPT_ORIGIN 0x00001000
+#define SUBOPT_MIN_SEND_DELAY 0x00002000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -90,6 +91,7 @@ typedef struct SubOpts
bool disableonerr;
char *origin;
XLogRecPtr lsn;
+ int32 min_send_delay;
} SubOpts;
static List *fetch_table_list(WalReceiverConn *wrconn, List *publications);
@@ -100,7 +102,7 @@ static void check_publications_origin(WalReceiverConn *wrconn,
static void check_duplicates_in_publist(List *publist, Datum *datums);
static List *merge_publications(List *oldpublist, List *newpublist, bool addpub, const char *subname);
static void ReportSlotConnectionError(List *rstates, Oid subid, char *slotname, char *err);
-
+static int32 defGetMinSendDelay(DefElem *def);
/*
* Common option parsing function for CREATE and ALTER SUBSCRIPTION commands.
@@ -146,6 +148,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->disableonerr = false;
if (IsSet(supported_opts, SUBOPT_ORIGIN))
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY))
+ opts->min_send_delay = 0;
/* Parse options */
foreach(lc, stmt_options)
@@ -324,6 +328,15 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->specified_opts |= SUBOPT_LSN;
opts->lsn = lsn;
}
+ else if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ if (IsSet(opts->specified_opts, SUBOPT_MIN_SEND_DELAY))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_MIN_SEND_DELAY;
+ opts->min_send_delay = defGetMinSendDelay(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -404,6 +417,29 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
"slot_name = NONE", "create_slot = false")));
}
}
+
+ /*
+ * The combination of parallel streaming mode and min_send_delay is not
+ * allowed. This is because in parallel streaming mode, the walsender
+ * starts sending the transaction stream without knowing the prepare/commit
+ * time of the transaction. Always waiting for the full 'min_send_delay'
+ * time to send may introduce unnecessary delay.
+ *
+ * The other possibility was to wait sending COMMIT record of the parallel
+ * apply transaction but that would cause issues related to resource bloat
+ * and locks being held for a long time.
+ */
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ opts->min_send_delay > 0 &&
+ opts->streaming == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ /*
+ * translator: the first %s is a string of the form "parameter > 0"
+ * and the second one is "option = value".
+ */
+ errmsg("%s and %s are mutually exclusive options",
+ "min_send_delay > 0", "streaming = parallel"));
}
/*
@@ -560,7 +596,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_SLOT_NAME | SUBOPT_COPY_DATA |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_TWOPHASE_COMMIT |
- SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN);
+ SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN |
+ SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -628,6 +665,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
values[Anum_pg_subscription_subname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->subname));
values[Anum_pg_subscription_subowner - 1] = ObjectIdGetDatum(owner);
+ values[Anum_pg_subscription_subminsenddelay - 1] = Int32GetDatum(opts.min_send_delay);
values[Anum_pg_subscription_subenabled - 1] = BoolGetDatum(opts.enabled);
values[Anum_pg_subscription_subbinary - 1] = BoolGetDatum(opts.binary);
values[Anum_pg_subscription_substream - 1] = CharGetDatum(opts.streaming);
@@ -1054,7 +1092,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
supported_opts = (SUBOPT_SLOT_NAME |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_DISABLE_ON_ERR |
- SUBOPT_ORIGIN);
+ SUBOPT_ORIGIN | SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options,
supported_opts, &opts);
@@ -1098,6 +1136,19 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
if (IsSet(opts.specified_opts, SUBOPT_STREAMING))
{
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.streaming == LOGICALREP_STREAM_PARALLEL &&
+ !IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY) &&
+ sub->minsenddelay > 0)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set parallel streaming mode for subscription with %s",
+ "min_send_delay"));
+
values[Anum_pg_subscription_substream - 1] =
CharGetDatum(opts.streaming);
replaces[Anum_pg_subscription_substream - 1] = true;
@@ -1111,6 +1162,26 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
= true;
}
+ if (IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY))
+ {
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.min_send_delay > 0 &&
+ !IsSet(opts.specified_opts, SUBOPT_STREAMING) &&
+ sub->stream == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set %s for subscription in parallel streaming mode",
+ "min_send_delay"));
+
+ values[Anum_pg_subscription_subminsenddelay - 1] =
+ Int32GetDatum(opts.min_send_delay);
+ replaces[Anum_pg_subscription_subminsenddelay - 1] = true;
+ }
+
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
{
values[Anum_pg_subscription_suborigin - 1] =
@@ -2195,3 +2266,45 @@ defGetStreamingMode(DefElem *def)
def->defname)));
return LOGICALREP_STREAM_OFF; /* keep compiler quiet */
}
+
+/*
+ * Extract the min_send_delay value from a DefElem. This is very similar to
+ * parse_and_validate_value() for integer values, because min_send_delay
+ * accepts the same parameter format as recovery_min_apply_delay.
+ */
+static int32
+defGetMinSendDelay(DefElem *def)
+{
+ char *input_string;
+ int result;
+ const char *hintmsg;
+
+ input_string = defGetString(def);
+
+ /*
+ * Parse given string as parameter which has millisecond unit
+ */
+ if (!parse_int(input_string, &result, GUC_UNIT_MS, &hintmsg))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid value for parameter \"%s\": \"%s\"",
+ "min_send_delay", input_string),
+ hintmsg ? errhint("%s", _(hintmsg)) : 0));
+
+ /*
+ * Check both the lower boundary for the valid min_send_delay range and
+ * the upper boundary as the safeguard for some platforms where INT_MAX is
+ * wider than int32 respectively. Although parse_int() has confirmed that
+ * the result is less than or equal to INT_MAX, the value will be stored
+ * in a catalog column of int32.
+ */
+ if (result < 0 || result > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%d ms is outside the valid range for parameter \"%s\" (%d .. %d)",
+ result,
+ "min_send_delay",
+ 0, PG_INT32_MAX)));
+
+ return result;
+}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 560ec974fa..89a72c1abe 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -443,6 +443,11 @@ libpqrcv_startstreaming(WalReceiverConn *conn,
PQserverVersion(conn->streamConn) >= 140000)
appendStringInfoString(&cmd, ", binary 'true'");
+ if (options->proto.logical.min_send_delay > 0 &&
+ PQserverVersion(conn->streamConn) >= 160000)
+ appendStringInfo(&cmd, ", min_send_delay '%d'",
+ options->proto.logical.min_send_delay);
+
appendStringInfoChar(&cmd, ')');
}
else
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index a53e23c679..2674444894 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -679,6 +679,15 @@ DecodeCommit(LogicalDecodingContext *ctx, XLogRecordBuffer *buf,
}
else
{
+ /*
+ * Delay sending the changes if required. For streaming transactions,
+ * this means a delay in sending the last stream but that is OK because
+ * on the downstream the changes will be applied only after receiving
+ * the last stream.
+ */
+ if (ctx->min_send_delay > 0 && ctx->delay_send)
+ ctx->delay_send(ctx, commit_time);
+
ReorderBufferCommit(ctx->reorder, xid, buf->origptr, buf->endptr,
commit_time, origin_id, origin_lsn);
}
@@ -763,6 +772,15 @@ DecodePrepare(LogicalDecodingContext *ctx, XLogRecordBuffer *buf,
buf->origptr, buf->endptr);
}
+ /*
+ * Delay sending the changes if required. For streaming transactions,
+ * this means a delay in sending the last stream but that is OK because
+ * on the downstream the changes will be applied only after receiving
+ * the last stream.
+ */
+ if (ctx->min_send_delay > 0 && ctx->delay_send)
+ ctx->delay_send(ctx, prepare_time);
+
/* replay actions of all transaction + subtransactions in order */
ReorderBufferPrepare(ctx->reorder, xid, parsed->twophase_gid);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c3ec97a0a6..e4dd822cdc 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -156,7 +156,8 @@ StartupDecodingContext(List *output_plugin_options,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send)
{
ReplicationSlot *slot;
MemoryContext context,
@@ -293,6 +294,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->prepare_write = prepare_write;
ctx->write = do_write;
ctx->update_progress = update_progress;
+ ctx->delay_send = delay_send;
ctx->output_plugin_options = output_plugin_options;
@@ -316,7 +318,7 @@ StartupDecodingContext(List *output_plugin_options,
* marking WAL reserved beforehand. In that scenario, it's up to the
* caller to guarantee that WAL remains available.
* xl_routine -- XLogReaderRoutine for underlying XLogReader
- * prepare_write, do_write, update_progress --
+ * prepare_write, do_write, update_progress, delay_send --
* callbacks that perform the use-case dependent, actual, work.
*
* Needs to be called while in a memory context that's at least as long lived
@@ -334,7 +336,8 @@ CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send)
{
TransactionId xmin_horizon = InvalidTransactionId;
ReplicationSlot *slot;
@@ -435,7 +438,7 @@ CreateInitDecodingContext(const char *plugin,
ctx = StartupDecodingContext(NIL, restart_lsn, xmin_horizon,
need_full_snapshot, false,
xl_routine, prepare_write, do_write,
- update_progress);
+ update_progress, delay_send);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
@@ -475,7 +478,7 @@ CreateInitDecodingContext(const char *plugin,
* xl_routine
* XLogReaderRoutine used by underlying xlogreader
*
- * prepare_write, do_write, update_progress
+ * prepare_write, do_write, update_progress, delay_send
* callbacks that have to be filled to perform the use-case dependent,
* actual work.
*
@@ -493,7 +496,8 @@ CreateDecodingContext(XLogRecPtr start_lsn,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send)
{
LogicalDecodingContext *ctx;
ReplicationSlot *slot;
@@ -547,7 +551,7 @@ CreateDecodingContext(XLogRecPtr start_lsn,
ctx = StartupDecodingContext(output_plugin_options,
start_lsn, InvalidTransactionId, false,
fast_forward, xl_routine, prepare_write,
- do_write, update_progress);
+ do_write, update_progress, delay_send);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c
index fa1b641a2b..960025197f 100644
--- a/src/backend/replication/logical/logicalfuncs.c
+++ b/src/backend/replication/logical/logicalfuncs.c
@@ -212,7 +212,7 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
LogicalOutputPrepareWrite,
- LogicalOutputWrite, NULL);
+ LogicalOutputWrite, NULL, NULL);
/*
* After the sanity checks in CreateDecodingContext, make sure the
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index cfb2ab6248..e68902ae34 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -3898,7 +3898,8 @@ maybe_reread_subscription(void)
newsub->stream != MySubscription->stream ||
strcmp(newsub->origin, MySubscription->origin) != 0 ||
newsub->owner != MySubscription->owner ||
- !equal(newsub->publications, MySubscription->publications))
+ !equal(newsub->publications, MySubscription->publications) ||
+ newsub->minsenddelay != MySubscription->minsenddelay)
{
if (am_parallel_apply_worker())
ereport(LOG,
@@ -4617,9 +4618,18 @@ ApplyWorkerMain(Datum main_arg)
options.proto.logical.twophase = false;
options.proto.logical.origin = pstrdup(MySubscription->origin);
+ options.proto.logical.min_send_delay = 0;
if (!am_tablesync_worker())
{
+ /*
+ * Time-delayed logical replication does not support tablesync
+ * workers, so only the leader apply worker can request walsenders to
+ * delay on the publisher side.
+ */
+ if (server_version >= 160000 && MySubscription->minsenddelay > 0)
+ options.proto.logical.min_send_delay = MySubscription->minsenddelay;
+
/*
* Even when the two_phase mode is requested by the user, it remains
* as the tri-state PENDING until all tablesyncs have reached READY
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 98377c094b..df6a87b0ba 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -285,6 +285,7 @@ parse_output_parameters(List *options, PGOutputData *data)
bool streaming_given = false;
bool two_phase_option_given = false;
bool origin_option_given = false;
+ bool min_send_delay_option_given = false;
data->binary = false;
data->streaming = LOGICALREP_STREAM_OFF;
@@ -396,6 +397,32 @@ parse_output_parameters(List *options, PGOutputData *data)
errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized origin value: \"%s\"", data->origin));
}
+ else if (strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ unsigned long delay_val;
+ char *endptr;
+
+ if (min_send_delay_option_given)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("conflicting or redundant options"));
+ min_send_delay_option_given = true;
+
+ errno = 0;
+ delay_val = strtoul(strVal(defel->arg), &endptr, 10);
+ if (errno != 0 || *endptr != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid min_send_delay")));
+
+ if (delay_val > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("min_send_delay \"%s\" out of range",
+ strVal(defel->arg))));
+
+ data->min_send_delay = (int32) delay_val;
+ }
else
elog(ERROR, "unrecognized pgoutput option: %s", defel->defname);
}
@@ -501,6 +528,15 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
else
ctx->twophase_opt_given = true;
+ if (data->min_send_delay &&
+ data->protocol_version < LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("requested proto_version=%d does not support delay sending data, need %d or higher",
+ data->protocol_version, LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM)));
+ else
+ ctx->min_send_delay = data->min_send_delay;
+
/* Init publication state. */
data->publications = NIL;
publications_valid = false;
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 2f3c964824..522f7600a1 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -148,7 +148,7 @@ create_logical_replication_slot(char *name, char *plugin,
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* If caller needs us to determine the decoding start point, do so now.
@@ -481,7 +481,7 @@ pg_logical_replication_slot_advance(XLogRecPtr moveto)
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* Start reading at the slot's restart_lsn, which we know to point to
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 75e8363e24..9537fba7df 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -252,6 +252,7 @@ static void WalSndPrepareWrite(LogicalDecodingContext *ctx, XLogRecPtr lsn, Tran
static void WalSndWriteData(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid, bool last_write);
static void WalSndUpdateProgress(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid,
bool skipped_xact);
+static void WalSndDelay(LogicalDecodingContext *ctx, TimestampTz delay_start);
static XLogRecPtr WalSndWaitForWal(XLogRecPtr loc);
static void LagTrackerWrite(XLogRecPtr lsn, TimestampTz local_flush_time);
static TimeOffset LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now);
@@ -1126,7 +1127,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
/*
* Signal that we don't need the timeout mechanism. We're just
@@ -1285,7 +1286,7 @@ StartLogicalReplication(StartReplicationCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
xlogreader = logical_decoding_ctx->reader;
WalSndSetState(WALSNDSTATE_CATCHUP);
@@ -3849,3 +3850,76 @@ LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now)
Assert(time != 0);
return now - time;
}
+
+/*
+ * LogicalDecodingContext 'delay' callback.
+ *
+ * Wait long enough to make sure a transaction is applied at least that
+ * period behind the publisher.
+ */
+static void
+WalSndDelay(LogicalDecodingContext *ctx, TimestampTz delay_start)
+{
+ /* Wait till delayUntil by the latch mechanism */
+ while (true)
+ {
+ TimestampTz delayUntil;
+ long remaining_wait_time_ms;
+ long timeout_sleeptime_ms;
+
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* This might change wal_sender_timeout */
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+
+ /* Check for input from the client */
+ ProcessRepliesIfAny();
+
+ /* die if timeout was reached */
+ WalSndCheckTimeOut();
+
+ /* Send keepalive if the time has come */
+ WalSndKeepaliveIfNecessary();
+
+ /* Try to flush pending output to the client */
+ if (pq_flush_if_writable() != 0)
+ WalSndShutdown();
+
+ /*
+ * If we've requested to shut down, exit the process.
+ *
+ * Note that WalSndDone() cannot be used here because the delaying
+ * changes will be sent in the function.
+ */
+ if (got_STOPPING)
+ WalSndShutdown();
+
+ delayUntil = TimestampTzPlusMilliseconds(delay_start, ctx->min_send_delay);
+ remaining_wait_time_ms = TimestampDifferenceMilliseconds(GetCurrentTimestamp(), delayUntil);
+
+ /*
+ * Exit without arming the latch if it's already past time to send
+ * this transaction.
+ */
+ if (remaining_wait_time_ms <= 0)
+ break;
+
+ /* Sleep until appropriate time. */
+ timeout_sleeptime_ms = WalSndComputeSleeptime(GetCurrentTimestamp());
+
+ elog(DEBUG2, "time-delayed replication for txid %u, delay_time = %d ms, remaining wait time: %ld ms",
+ ctx->write_xid, (int) ctx->min_send_delay,
+ remaining_wait_time_ms);
+
+ /* Sleep until we get reply from worker or we time out */
+ WalSndWait(WL_SOCKET_READABLE,
+ Min(timeout_sleeptime_ms, remaining_wait_time_ms),
+ WAIT_EVENT_WALSENDER_SEND_DELAY);
+ }
+}
diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c
index cb99cc6339..76c19fe11d 100644
--- a/src/backend/utils/activity/wait_event.c
+++ b/src/backend/utils/activity/wait_event.c
@@ -515,6 +515,9 @@ pgstat_get_wait_timeout(WaitEventTimeout w)
case WAIT_EVENT_VACUUM_TRUNCATE:
event_name = "VacuumTruncate";
break;
+ case WAIT_EVENT_WALSENDER_SEND_DELAY:
+ event_name = "WalSenderSendDelay";
+ break;
/* no default case, so that compiler will warn */
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1a06eeaf6a..9754487921 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4494,6 +4494,7 @@ getSubscriptions(Archive *fout)
int i_subsynccommit;
int i_subpublications;
int i_subbinary;
+ int i_subminsenddelay;
int i,
ntups;
@@ -4546,9 +4547,13 @@ getSubscriptions(Archive *fout)
LOGICALREP_TWOPHASE_STATE_DISABLED);
if (fout->remoteVersion >= 160000)
- appendPQExpBufferStr(query, " s.suborigin\n");
+ appendPQExpBufferStr(query,
+ " s.suborigin,\n"
+ " s.subminsenddelay\n");
else
- appendPQExpBuffer(query, " '%s' AS suborigin\n", LOGICALREP_ORIGIN_ANY);
+ appendPQExpBuffer(query, " '%s' AS suborigin,\n"
+ " 0 AS subminsenddelay\n",
+ LOGICALREP_ORIGIN_ANY);
appendPQExpBufferStr(query,
"FROM pg_subscription s\n"
@@ -4576,6 +4581,7 @@ getSubscriptions(Archive *fout)
i_subtwophasestate = PQfnumber(res, "subtwophasestate");
i_subdisableonerr = PQfnumber(res, "subdisableonerr");
i_suborigin = PQfnumber(res, "suborigin");
+ i_subminsenddelay = PQfnumber(res, "subminsenddelay");
subinfo = pg_malloc(ntups * sizeof(SubscriptionInfo));
@@ -4606,6 +4612,8 @@ getSubscriptions(Archive *fout)
subinfo[i].subdisableonerr =
pg_strdup(PQgetvalue(res, i, i_subdisableonerr));
subinfo[i].suborigin = pg_strdup(PQgetvalue(res, i, i_suborigin));
+ subinfo[i].subminsenddelay =
+ atoi(PQgetvalue(res, i, i_subminsenddelay));
/* Decide whether we want to dump it */
selectDumpableObject(&(subinfo[i].dobj), fout);
@@ -4687,6 +4695,9 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
+ if (subinfo->subminsenddelay > 0)
+ appendPQExpBuffer(query, ", min_send_delay = '%d ms'", subinfo->subminsenddelay);
+
appendPQExpBufferStr(query, ");\n");
if (subinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index cdca0b993d..4c55f8efc4 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -660,6 +660,7 @@ typedef struct _SubscriptionInfo
char *subdisableonerr;
char *suborigin;
char *subsynccommit;
+ int subminsenddelay;
char *subpublications;
} SubscriptionInfo;
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index c8a0bb7b3a..c7d303a168 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6472,7 +6472,7 @@ describeSubscriptions(const char *pattern, bool verbose)
PGresult *res;
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
- false, false, false, false, false, false, false, false};
+ false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6527,10 +6527,13 @@ describeSubscriptions(const char *pattern, bool verbose)
gettext_noop("Two-phase commit"),
gettext_noop("Disable on error"));
+ /* Origin and min_send_delay are only supported in v16 and higher */
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
- ", suborigin AS \"%s\"\n",
- gettext_noop("Origin"));
+ ", suborigin AS \"%s\"\n"
+ ", subminsenddelay AS \"%s\"\n",
+ gettext_noop("Origin"),
+ gettext_noop("Min send delay"));
appendPQExpBuffer(&buf,
", subsynccommit AS \"%s\"\n"
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5e1882eaea..6643db6f55 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1925,7 +1925,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("(", "PUBLICATION");
/* ALTER SUBSCRIPTION <name> SET ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SET", "("))
- COMPLETE_WITH("binary", "disable_on_error", "origin", "slot_name",
+ COMPLETE_WITH("binary", "disable_on_error", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit");
/* ALTER SUBSCRIPTION <name> SKIP ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SKIP", "("))
@@ -3268,7 +3268,7 @@ psql_completion(const char *text, int start, int end)
/* Complete "CREATE SUBSCRIPTION <name> ... WITH ( <opt>" */
else if (HeadMatches("CREATE", "SUBSCRIPTION") && TailMatches("WITH", "("))
COMPLETE_WITH("binary", "connect", "copy_data", "create_slot",
- "disable_on_error", "enabled", "origin", "slot_name",
+ "disable_on_error", "enabled", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit", "two_phase");
/* CREATE TRIGGER --- is allowed inside CREATE SCHEMA, so use TailMatches */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index b0f2a1705d..69ae4314b4 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -74,6 +74,8 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
Oid subowner BKI_LOOKUP(pg_authid); /* Owner of the subscription */
+ int32 subminsenddelay; /* Replication send delay (ms) */
+
bool subenabled; /* True if the subscription is enabled (the
* worker should be running) */
@@ -122,6 +124,7 @@ typedef struct Subscription
* skipped */
char *name; /* Name of the subscription */
Oid owner; /* Oid of the subscription owner */
+ int32 minsenddelay; /* Replication send delay (ms) */
bool enabled; /* Indicates if the subscription is enabled */
bool binary; /* Indicates if the subscription wants data in
* binary format */
diff --git a/src/include/replication/logical.h b/src/include/replication/logical.h
index 5f49554ea0..b0770edf49 100644
--- a/src/include/replication/logical.h
+++ b/src/include/replication/logical.h
@@ -30,6 +30,10 @@ typedef void (*LogicalOutputPluginWriterUpdateProgress) (struct LogicalDecodingC
bool skipped_xact
);
+typedef void (*LogicalOutputPluginWriterDelay) (struct LogicalDecodingContext *lr,
+ TimestampTz start_time
+);
+
typedef struct LogicalDecodingContext
{
/* memory context this is all allocated in */
@@ -64,6 +68,7 @@ typedef struct LogicalDecodingContext
LogicalOutputPluginWriterPrepareWrite prepare_write;
LogicalOutputPluginWriterWrite write;
LogicalOutputPluginWriterUpdateProgress update_progress;
+ LogicalOutputPluginWriterDelay delay_send;
/*
* Output buffer.
@@ -100,6 +105,12 @@ typedef struct LogicalDecodingContext
*/
bool twophase_opt_given;
+ /*
+ * The minimum delay, in milliseconds, by the publisher before sending all
+ * the changes
+ */
+ int32 min_send_delay;
+
/*
* State for writing output.
*/
@@ -121,14 +132,16 @@ extern LogicalDecodingContext *CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send);
extern LogicalDecodingContext *CreateDecodingContext(XLogRecPtr start_lsn,
List *output_plugin_options,
bool fast_forward,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send);
extern void DecodingContextFindStartpoint(LogicalDecodingContext *ctx);
extern bool DecodingContextReady(LogicalDecodingContext *ctx);
extern void FreeDecodingContext(LogicalDecodingContext *ctx);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index 0ea2df5088..46faadbd7a 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -36,13 +36,17 @@
* LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM is the minimum protocol version
* where we support applying large streaming transactions in parallel.
* Introduced in PG16.
+ *
+ * LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM is the minimum protocol version
+ * with support for delaying to send transactions. Introduced in PG16.
*/
#define LOGICALREP_PROTO_MIN_VERSION_NUM 1
#define LOGICALREP_PROTO_VERSION_NUM 1
#define LOGICALREP_PROTO_STREAM_VERSION_NUM 2
#define LOGICALREP_PROTO_TWOPHASE_VERSION_NUM 3
#define LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM 4
-#define LOGICALREP_PROTO_MAX_VERSION_NUM LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM
+#define LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM 4
+#define LOGICALREP_PROTO_MAX_VERSION_NUM LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM
/*
* Logical message types
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index b4a8015403..d2fde09e00 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -30,6 +30,7 @@ typedef struct PGOutputData
bool messages;
bool two_phase;
char *origin;
+ int32 min_send_delay;
} PGOutputData;
#endif /* PGOUTPUT_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index decffe352d..c20969aed7 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -187,6 +187,7 @@ typedef struct
* prepare time */
char *origin; /* Only publish data originating from the
* specified origin */
+ int32 min_send_delay; /* The minimum send delay */
} logical;
} proto;
} WalRcvStreamOptions;
diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h
index 9ab23e1c4a..cc3a234eba 100644
--- a/src/include/utils/wait_event.h
+++ b/src/include/utils/wait_event.h
@@ -150,7 +150,8 @@ typedef enum
WAIT_EVENT_REGISTER_SYNC_REQUEST,
WAIT_EVENT_SPIN_DELAY,
WAIT_EVENT_VACUUM_DELAY,
- WAIT_EVENT_VACUUM_TRUNCATE
+ WAIT_EVENT_VACUUM_TRUNCATE,
+ WAIT_EVENT_WALSENDER_SEND_DELAY
} WaitEventTimeout;
/* ----------
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 3f99b14394..2027316233 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -114,18 +114,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -143,10 +143,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -163,10 +163,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/12345
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/12345
(1 row)
-- ok - with lsn = NONE
@@ -175,10 +175,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
BEGIN;
@@ -210,10 +210,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
ERROR: invalid value for parameter "synchronous_commit": "foobar"
HINT: Available values: local, remote_write, remote_apply, on, off.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | local | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | local | dbname=regress_doesnotexist2 | 0/0
(1 row)
-- rename back to keep the rest simple
@@ -247,19 +247,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -271,27 +271,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication already exists
@@ -306,10 +306,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication used more than once
@@ -324,10 +324,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -363,10 +363,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
--fail - alter of two_phase option not supported.
@@ -375,10 +375,10 @@ ERROR: unrecognized subscription parameter: "two_phase"
-- but can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -388,10 +388,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -404,20 +404,61 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | 0 | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+ERROR: invalid value for parameter "min_send_delay": "foo"
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+ERROR: -1 ms is outside the valid range for parameter "min_send_delay" (0 .. 2147483647)
+-- fail - specifying streaming = parallel with time-delayed replication is not
+-- supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+ERROR: min_send_delay > 0 and streaming = parallel are mutually exclusive options
+-- success -- min_send_delay value without units is taken as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 123 | off | dbname=regress_doesnotexit | 0/0
+(1 row)
+
+-- success -- min_send_delay value with units other than ms is converted to ms
+-- and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 86400000 | off | dbname=regress_doesnotexit | 0/0
(1 row)
+-- fail - alter subscription with streaming = parallel should fail when
+-- time-delayed replication is set
+ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
+ERROR: cannot set parallel streaming mode for subscription with min_send_delay
+-- fail - alter subscription with min_send_delay should fail when
+-- streaming = parallel is set
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 0, streaming = parallel);
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 123);
+ERROR: cannot set min_send_delay for subscription in parallel streaming mode
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 7281f5fee2..46bf4a27d9 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -286,6 +286,35 @@ ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+
+-- fail - specifying streaming = parallel with time-delayed replication is not
+-- supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+
+-- success -- min_send_delay value without units is taken as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+\dRs+
+
+-- success -- min_send_delay value with units other than ms is converted to ms
+-- and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+
+-- fail - alter subscription with streaming = parallel should fail when
+-- time-delayed replication is set
+ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
+
+-- fail - alter subscription with min_send_delay should fail when
+-- streaming = parallel is set
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 0, streaming = parallel);
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 123);
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
RESET SESSION AUTHORIZATION;
DROP ROLE regress_subscription_user;
DROP ROLE regress_subscription_user2;
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 91aa068c95..063a98fde9 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -515,6 +515,33 @@ $node_publisher->poll_query_until('postgres',
or die
"Timed out while waiting for apply to restart after renaming SUBSCRIPTION";
+# Test time-delayed logical replication
+#
+# If the subscription sets min_send_delay parameter, the walsender will delay
+# the transaction send for min_send_delay milliseconds. We verify this by
+# looking at the time difference between a) when tuples are inserted on the
+# publisher, and b) when those changes are replicated on the subscriber. Even
+# on slow machines, this strategy will give predictable behavior.
+
+# Set min_send_delay parameter to 3 seconds
+my $delay = 3;
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub_renamed SET (min_send_delay = '${delay}s')");
+
+# Before doing the insertion, get the current timestamp that will be
+# used as a comparison base.
+my $publisher_insert_time = time();
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_ins VALUES (generate_series(1101, 1120))");
+
+# The publisher waits for the replication to complete
+$node_publisher->wait_for_catchup('tap_sub_renamed');
+
+# This test is successful only if at least the configured delay has elapsed.
+ok( time() - $publisher_insert_time >= $delay,
+ "subscriber applies WAL only after replication delay for non-streaming transaction"
+);
+
# check all the cleanup
$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub_renamed");
--
2.27.0
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-23 01:19 Peter Smith <[email protected]>
parent: Hayato Kuroda (Fujitsu) <[email protected]>
1 sibling, 0 replies; 17+ messages in thread
From: Peter Smith @ 2023-02-23 01:19 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Amit Kapila <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Patch v6 LGTM.
------
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-23 08:55 [email protected] <[email protected]>
parent: Hayato Kuroda (Fujitsu) <[email protected]>
1 sibling, 1 reply; 17+ messages in thread
From: [email protected] @ 2023-02-23 08:55 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; 'Amit Kapila' <[email protected]>; +Cc: Peter Smith <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
On Wed, Feb 22, 2023 9:48 PM Kuroda, Hayato/黒田 隼人 <[email protected]> wrote:
>
> Thank you for reviewing! PSA new version.
>
Thanks for your patch. Here is a comment.
+ elog(DEBUG2, "time-delayed replication for txid %u, delay_time = %d ms, remaining wait time: %ld ms",
+ ctx->write_xid, (int) ctx->min_send_delay,
+ remaining_wait_time_ms);
I tried and saw that the xid here looks wrong, what it got is the xid of the
previous transaction. It seems `ctx->write_xid` has not been updated and we
can't use it.
Regards,
Shi Yu
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-23 12:09 Hayato Kuroda (Fujitsu) <[email protected]>
parent: [email protected] <[email protected]>
0 siblings, 1 reply; 17+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2023-02-23 12:09 UTC (permalink / raw)
To: [email protected] <[email protected]>; +Cc: Peter Smith <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; 'Amit Kapila' <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Dear Shi,
Thank you for reviewing! PSA new version.
> + elog(DEBUG2, "time-delayed replication for txid %u, delay_time
> = %d ms, remaining wait time: %ld ms",
> + ctx->write_xid, (int) ctx->min_send_delay,
> + remaining_wait_time_ms);
>
> I tried and saw that the xid here looks wrong, what it got is the xid of the
> previous transaction. It seems `ctx->write_xid` has not been updated and we
> can't use it.
>
Good catch. There are several approaches to fix that, I choose the simplest way.
TransactionId was added as an argument of functions.
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
Attachments:
[application/octet-stream] v7-0001-Time-delayed-logical-replication-on-publisher-sid.patch (82.6K, ../../TYAPR01MB5866F85511EE35513A899AC3F5AB9@TYAPR01MB5866.jpnprd01.prod.outlook.com/2-v7-0001-Time-delayed-logical-replication-on-publisher-sid.patch)
download | inline diff:
From d4b9a056139c8fccc7cdc73d26ef6dd804ae5564 Mon Sep 17 00:00:00 2001
From: Hayato Kuroda <[email protected]>
Date: Thu, 16 Feb 2023 07:52:23 +0000
Subject: [PATCH v7] Time-delayed logical replication on publisher side
Similar to physical replication, a time-delayed copy of the data for
logical replication is useful for some scenarios (particularly to fix
errors that might cause data loss).
This patch implements a new subscription parameter called 'min_send_delay'.
If the subscription sets min_send_delay parameter, the apply worker
(via walrcv_startstreaming) passes the value to the publisher as an output plugin
option. The walsender will delay the transaction sending for given milliseconds.
The delay does not take into account the overhead of time spent in transferring
the transaction, which means that the arrival time at the subscriber may be
delayed more than the given time.
The delay occurs before we start to send the transaction on the publisher.
Regular and prepared transactions are covered. Streamed transactions are also
covered.
The combination of parallel streaming mode and min_send_delay is not allowed
1. This is because in parallel streaming mode, we start applying the transaction
stream as soon as the first change arrives without knowing the transaction's
prepare/commit time. Always waiting for the full 'min_send_delay' period might
include unnecessary delay.
2. Another reason is, for parallel streaming, the transaction will be opened
immediately by the parallel apply worker. So if walsender delayed to send the
final record of the transaction, the parallel worker must wait receiving with an
opened transaction. This would lead that locks acquired during the transaction
not getting released till min_send_delay elapsed.
Earlier versions were written by Euler Taveira, Takamichi Osumi, and Kuroda Hayato
Author: Kuroda Hayato, Takamichi Osumi
Reviewed-by: Amit Kapila, Peter Smith, Vignesh C, Shveta Malik,
Kyotaro Horiguchi, Shi Yu, Wang Wei, Dilip Kumar, Melih Mutlu,
Andres Freund
---
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/glossary.sgml | 15 ++
doc/src/sgml/logical-replication.sgml | 6 +
doc/src/sgml/monitoring.sgml | 5 +
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 43 +++-
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 7 +-
src/backend/commands/subscriptioncmds.c | 119 ++++++++++-
.../libpqwalreceiver/libpqwalreceiver.c | 5 +
src/backend/replication/logical/decode.c | 18 ++
src/backend/replication/logical/logical.c | 18 +-
.../replication/logical/logicalfuncs.c | 2 +-
src/backend/replication/logical/worker.c | 12 +-
src/backend/replication/pgoutput/pgoutput.c | 36 ++++
src/backend/replication/slotfuncs.c | 4 +-
src/backend/replication/walsender.c | 77 +++++++-
src/backend/utils/activity/wait_event.c | 3 +
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 9 +-
src/bin/psql/tab-complete.c | 4 +-
src/include/catalog/pg_subscription.h | 3 +
src/include/replication/logical.h | 18 +-
src/include/replication/logicalproto.h | 6 +-
src/include/replication/pgoutput.h | 1 +
src/include/replication/walreceiver.h | 1 +
src/include/utils/wait_event.h | 3 +-
src/test/regress/expected/subscription.out | 185 +++++++++++-------
src/test/regress/sql/subscription.sql | 29 +++
src/test/subscription/t/001_rep_changes.pl | 27 +++
31 files changed, 583 insertions(+), 105 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c1e4048054..ce6521fe6c 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -7892,6 +7892,16 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subminsenddelay</structfield> <type>int4</type>
+ </para>
+ <para>
+ The minimum delay, in milliseconds, by the publisher before sending all
+ the changes
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subenabled</structfield> <type>bool</type>
diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml
index 7c01a541fe..9ede9d05f6 100644
--- a/doc/src/sgml/glossary.sgml
+++ b/doc/src/sgml/glossary.sgml
@@ -1729,6 +1729,21 @@
</glossdef>
</glossentry>
+ <glossentry id="glossary-time-delayed-replication">
+ <glossterm>Time-delayed replication</glossterm>
+ <glossdef>
+ <para>
+ Replication setup that delays the application of changes by a specified
+ minimum time-delay period.
+ </para>
+ <para>
+ For more information, see
+ <xref linkend="guc-recovery-min-apply-delay"/> for physical replication
+ and <xref linkend="sql-createsubscription"/> for logical replication.
+ </para>
+ </glossdef>
+ </glossentry>
+
<glossentry id="glossary-toast">
<glossterm>TOAST</glossterm>
<glossdef>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 1bd5660c87..fe9e7f7b26 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -247,6 +247,12 @@
target table.
</para>
+ <para>
+ A subscription can delay the receipt of changes by specifying the
+ <literal>min_send_delay</literal> subscription parameter. See
+ <xref linkend="sql-createsubscription"/> for details.
+ </para>
+
<sect2 id="logical-replication-subscription-slot">
<title>Replication Slot Management</title>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b0b997f092..6158587644 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -2349,6 +2349,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry>Waiting to acquire an exclusive lock to truncate off any
empty pages at the end of a table vacuumed.</entry>
</row>
+ <row>
+ <entry><literal>WalSenderSendDelay</literal></entry>
+ <entry>Waiting while sending changes for time-delayed logical replication
+ in the WAL sender process.</entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 964fcbb8ff..3f238b958b 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -213,8 +213,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
are <literal>slot_name</literal>,
<literal>synchronous_commit</literal>,
<literal>binary</literal>, <literal>streaming</literal>,
- <literal>disable_on_error</literal>, and
- <literal>origin</literal>.
+ <literal>disable_on_error</literal>,
+ <literal>origin</literal>, and
+ <literal>min_send_delay</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 51c45f17c7..9d08740ba2 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -349,7 +349,43 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
- </variablelist></para>
+
+ <varlistentry>
+ <term><literal>min_send_delay</literal> (<type>integer</type>)</term>
+ <listitem>
+ <para>
+ By default, the publisher sends changes as soon as possible. This
+ parameter allows the user to delay changes by the given time period.
+ If the value is specified without units, it is taken as milliseconds.
+ The default is zero (no delay). See <xref linkend="config-setting-names-values"/>
+ for details on the available valid time units.
+ </para>
+ <para>
+ The delay is effective only when the initial table synchronization
+ has been finished. However, there is a possibility that the table
+ status updated in <link linkend="catalog-pg-subscription-rel"><structname>pg_subscription_rel</structname></link>
+ could be delayed in getting to the "ready" state, and also two-phase
+ (if specified) could be delayed in getting to "enabled".
+ </para>
+ <para>
+ The delay does not take into account the overhead of time spent
+ transferring the transaction. Therefore, the arrival time at the
+ subscriber may be delayed more than the specified
+ <literal>min_send_delay</literal> time.
+ </para>
+ <warning>
+ <para>
+ Delaying the replication means there is a much longer time between
+ making a change on the publisher, and that change being committed
+ on the subscriber. This can impact the performance of synchronous
+ replication. See <xref linkend="guc-synchronous-commit"/>
+ parameter.
+ </para>
+ </warning>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
</listitem>
</varlistentry>
@@ -420,6 +456,11 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
published with different column lists are not supported.
</para>
+ <para>
+ A non-zero <literal>min_send_delay</literal> parameter is not allowed when
+ streaming in parallel mode.
+ </para>
+
<para>
We allow non-existent publications to be specified so that users can add
those later. This means
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index a56ae311c3..63a10b06d1 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -66,6 +66,7 @@ GetSubscription(Oid subid, bool missing_ok)
sub->skiplsn = subform->subskiplsn;
sub->name = pstrdup(NameStr(subform->subname));
sub->owner = subform->subowner;
+ sub->minsenddelay = subform->subminsenddelay;
sub->enabled = subform->subenabled;
sub->binary = subform->subbinary;
sub->stream = subform->substream;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 34ca0e739f..54a705d71b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1314,9 +1314,10 @@ REVOKE ALL ON pg_replication_origin_status FROM public;
-- All columns of pg_subscription except subconninfo are publicly readable.
REVOKE ALL ON pg_subscription FROM public;
-GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
- subbinary, substream, subtwophasestate, subdisableonerr,
- subslotname, subsynccommit, subpublications, suborigin)
+GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subminsenddelay,
+ subenabled, subbinary, substream, subtwophasestate,
+ subdisableonerr, subslotname, subsynccommit, subpublications,
+ suborigin)
ON pg_subscription TO public;
CREATE VIEW pg_stat_subscription_stats AS
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 464db6d247..4a8cd47171 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -66,6 +66,7 @@
#define SUBOPT_DISABLE_ON_ERR 0x00000400
#define SUBOPT_LSN 0x00000800
#define SUBOPT_ORIGIN 0x00001000
+#define SUBOPT_MIN_SEND_DELAY 0x00002000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -90,6 +91,7 @@ typedef struct SubOpts
bool disableonerr;
char *origin;
XLogRecPtr lsn;
+ int32 min_send_delay;
} SubOpts;
static List *fetch_table_list(WalReceiverConn *wrconn, List *publications);
@@ -100,7 +102,7 @@ static void check_publications_origin(WalReceiverConn *wrconn,
static void check_duplicates_in_publist(List *publist, Datum *datums);
static List *merge_publications(List *oldpublist, List *newpublist, bool addpub, const char *subname);
static void ReportSlotConnectionError(List *rstates, Oid subid, char *slotname, char *err);
-
+static int32 defGetMinSendDelay(DefElem *def);
/*
* Common option parsing function for CREATE and ALTER SUBSCRIPTION commands.
@@ -146,6 +148,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->disableonerr = false;
if (IsSet(supported_opts, SUBOPT_ORIGIN))
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY))
+ opts->min_send_delay = 0;
/* Parse options */
foreach(lc, stmt_options)
@@ -324,6 +328,15 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->specified_opts |= SUBOPT_LSN;
opts->lsn = lsn;
}
+ else if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ if (IsSet(opts->specified_opts, SUBOPT_MIN_SEND_DELAY))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_MIN_SEND_DELAY;
+ opts->min_send_delay = defGetMinSendDelay(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -404,6 +417,29 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
"slot_name = NONE", "create_slot = false")));
}
}
+
+ /*
+ * The combination of parallel streaming mode and min_send_delay is not
+ * allowed. This is because in parallel streaming mode, the walsender
+ * starts sending the transaction stream without knowing the prepare/commit
+ * time of the transaction. Always waiting for the full 'min_send_delay'
+ * time to send may introduce unnecessary delay.
+ *
+ * The other possibility was to wait sending COMMIT record of the parallel
+ * apply transaction but that would cause issues related to resource bloat
+ * and locks being held for a long time.
+ */
+ if (IsSet(supported_opts, SUBOPT_MIN_SEND_DELAY) &&
+ opts->min_send_delay > 0 &&
+ opts->streaming == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ /*
+ * translator: the first %s is a string of the form "parameter > 0"
+ * and the second one is "option = value".
+ */
+ errmsg("%s and %s are mutually exclusive options",
+ "min_send_delay > 0", "streaming = parallel"));
}
/*
@@ -560,7 +596,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_SLOT_NAME | SUBOPT_COPY_DATA |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_TWOPHASE_COMMIT |
- SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN);
+ SUBOPT_DISABLE_ON_ERR | SUBOPT_ORIGIN |
+ SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -628,6 +665,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
values[Anum_pg_subscription_subname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->subname));
values[Anum_pg_subscription_subowner - 1] = ObjectIdGetDatum(owner);
+ values[Anum_pg_subscription_subminsenddelay - 1] = Int32GetDatum(opts.min_send_delay);
values[Anum_pg_subscription_subenabled - 1] = BoolGetDatum(opts.enabled);
values[Anum_pg_subscription_subbinary - 1] = BoolGetDatum(opts.binary);
values[Anum_pg_subscription_substream - 1] = CharGetDatum(opts.streaming);
@@ -1054,7 +1092,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
supported_opts = (SUBOPT_SLOT_NAME |
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_DISABLE_ON_ERR |
- SUBOPT_ORIGIN);
+ SUBOPT_ORIGIN | SUBOPT_MIN_SEND_DELAY);
parse_subscription_options(pstate, stmt->options,
supported_opts, &opts);
@@ -1098,6 +1136,19 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
if (IsSet(opts.specified_opts, SUBOPT_STREAMING))
{
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.streaming == LOGICALREP_STREAM_PARALLEL &&
+ !IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY) &&
+ sub->minsenddelay > 0)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set parallel streaming mode for subscription with %s",
+ "min_send_delay"));
+
values[Anum_pg_subscription_substream - 1] =
CharGetDatum(opts.streaming);
replaces[Anum_pg_subscription_substream - 1] = true;
@@ -1111,6 +1162,26 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
= true;
}
+ if (IsSet(opts.specified_opts, SUBOPT_MIN_SEND_DELAY))
+ {
+ /*
+ * The combination of parallel streaming mode and
+ * min_send_delay is not allowed. See
+ * parse_subscription_options.
+ */
+ if (opts.min_send_delay > 0 &&
+ !IsSet(opts.specified_opts, SUBOPT_STREAMING) &&
+ sub->stream == LOGICALREP_STREAM_PARALLEL)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set %s for subscription in parallel streaming mode",
+ "min_send_delay"));
+
+ values[Anum_pg_subscription_subminsenddelay - 1] =
+ Int32GetDatum(opts.min_send_delay);
+ replaces[Anum_pg_subscription_subminsenddelay - 1] = true;
+ }
+
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
{
values[Anum_pg_subscription_suborigin - 1] =
@@ -2195,3 +2266,45 @@ defGetStreamingMode(DefElem *def)
def->defname)));
return LOGICALREP_STREAM_OFF; /* keep compiler quiet */
}
+
+/*
+ * Extract the min_send_delay value from a DefElem. This is very similar to
+ * parse_and_validate_value() for integer values, because min_send_delay
+ * accepts the same parameter format as recovery_min_apply_delay.
+ */
+static int32
+defGetMinSendDelay(DefElem *def)
+{
+ char *input_string;
+ int result;
+ const char *hintmsg;
+
+ input_string = defGetString(def);
+
+ /*
+ * Parse given string as parameter which has millisecond unit
+ */
+ if (!parse_int(input_string, &result, GUC_UNIT_MS, &hintmsg))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid value for parameter \"%s\": \"%s\"",
+ "min_send_delay", input_string),
+ hintmsg ? errhint("%s", _(hintmsg)) : 0));
+
+ /*
+ * Check both the lower boundary for the valid min_send_delay range and
+ * the upper boundary as the safeguard for some platforms where INT_MAX is
+ * wider than int32 respectively. Although parse_int() has confirmed that
+ * the result is less than or equal to INT_MAX, the value will be stored
+ * in a catalog column of int32.
+ */
+ if (result < 0 || result > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%d ms is outside the valid range for parameter \"%s\" (%d .. %d)",
+ result,
+ "min_send_delay",
+ 0, PG_INT32_MAX)));
+
+ return result;
+}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 560ec974fa..89a72c1abe 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -443,6 +443,11 @@ libpqrcv_startstreaming(WalReceiverConn *conn,
PQserverVersion(conn->streamConn) >= 140000)
appendStringInfoString(&cmd, ", binary 'true'");
+ if (options->proto.logical.min_send_delay > 0 &&
+ PQserverVersion(conn->streamConn) >= 160000)
+ appendStringInfo(&cmd, ", min_send_delay '%d'",
+ options->proto.logical.min_send_delay);
+
appendStringInfoChar(&cmd, ')');
}
else
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 8fe7bb65f1..31bf43bd63 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -689,6 +689,15 @@ DecodeCommit(LogicalDecodingContext *ctx, XLogRecordBuffer *buf,
}
else
{
+ /*
+ * Delay sending the changes if required. For streaming transactions,
+ * this means a delay in sending the last stream but that is OK
+ * because on the downstream the changes will be applied only after
+ * receiving the last stream.
+ */
+ if (ctx->min_send_delay > 0 && ctx->delay_send)
+ ctx->delay_send(ctx, xid, commit_time);
+
ReorderBufferCommit(ctx->reorder, xid, buf->origptr, buf->endptr,
commit_time, origin_id, origin_lsn);
}
@@ -773,6 +782,15 @@ DecodePrepare(LogicalDecodingContext *ctx, XLogRecordBuffer *buf,
buf->origptr, buf->endptr);
}
+ /*
+ * Delay sending the changes if required. For streaming transactions, this
+ * means a delay in sending the last stream but that is OK because on the
+ * downstream the changes will be applied only after receiving the last
+ * stream.
+ */
+ if (ctx->min_send_delay > 0 && ctx->delay_send)
+ ctx->delay_send(ctx, xid, prepare_time);
+
/* replay actions of all transaction + subtransactions in order */
ReorderBufferPrepare(ctx->reorder, xid, parsed->twophase_gid);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c3ec97a0a6..e4dd822cdc 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -156,7 +156,8 @@ StartupDecodingContext(List *output_plugin_options,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send)
{
ReplicationSlot *slot;
MemoryContext context,
@@ -293,6 +294,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->prepare_write = prepare_write;
ctx->write = do_write;
ctx->update_progress = update_progress;
+ ctx->delay_send = delay_send;
ctx->output_plugin_options = output_plugin_options;
@@ -316,7 +318,7 @@ StartupDecodingContext(List *output_plugin_options,
* marking WAL reserved beforehand. In that scenario, it's up to the
* caller to guarantee that WAL remains available.
* xl_routine -- XLogReaderRoutine for underlying XLogReader
- * prepare_write, do_write, update_progress --
+ * prepare_write, do_write, update_progress, delay_send --
* callbacks that perform the use-case dependent, actual, work.
*
* Needs to be called while in a memory context that's at least as long lived
@@ -334,7 +336,8 @@ CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send)
{
TransactionId xmin_horizon = InvalidTransactionId;
ReplicationSlot *slot;
@@ -435,7 +438,7 @@ CreateInitDecodingContext(const char *plugin,
ctx = StartupDecodingContext(NIL, restart_lsn, xmin_horizon,
need_full_snapshot, false,
xl_routine, prepare_write, do_write,
- update_progress);
+ update_progress, delay_send);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
@@ -475,7 +478,7 @@ CreateInitDecodingContext(const char *plugin,
* xl_routine
* XLogReaderRoutine used by underlying xlogreader
*
- * prepare_write, do_write, update_progress
+ * prepare_write, do_write, update_progress, delay_send
* callbacks that have to be filled to perform the use-case dependent,
* actual work.
*
@@ -493,7 +496,8 @@ CreateDecodingContext(XLogRecPtr start_lsn,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress)
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send)
{
LogicalDecodingContext *ctx;
ReplicationSlot *slot;
@@ -547,7 +551,7 @@ CreateDecodingContext(XLogRecPtr start_lsn,
ctx = StartupDecodingContext(output_plugin_options,
start_lsn, InvalidTransactionId, false,
fast_forward, xl_routine, prepare_write,
- do_write, update_progress);
+ do_write, update_progress, delay_send);
/* call output plugin initialization callback */
old_context = MemoryContextSwitchTo(ctx->context);
diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c
index fa1b641a2b..960025197f 100644
--- a/src/backend/replication/logical/logicalfuncs.c
+++ b/src/backend/replication/logical/logicalfuncs.c
@@ -212,7 +212,7 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
LogicalOutputPrepareWrite,
- LogicalOutputWrite, NULL);
+ LogicalOutputWrite, NULL, NULL);
/*
* After the sanity checks in CreateDecodingContext, make sure the
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index cfb2ab6248..e68902ae34 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -3898,7 +3898,8 @@ maybe_reread_subscription(void)
newsub->stream != MySubscription->stream ||
strcmp(newsub->origin, MySubscription->origin) != 0 ||
newsub->owner != MySubscription->owner ||
- !equal(newsub->publications, MySubscription->publications))
+ !equal(newsub->publications, MySubscription->publications) ||
+ newsub->minsenddelay != MySubscription->minsenddelay)
{
if (am_parallel_apply_worker())
ereport(LOG,
@@ -4617,9 +4618,18 @@ ApplyWorkerMain(Datum main_arg)
options.proto.logical.twophase = false;
options.proto.logical.origin = pstrdup(MySubscription->origin);
+ options.proto.logical.min_send_delay = 0;
if (!am_tablesync_worker())
{
+ /*
+ * Time-delayed logical replication does not support tablesync
+ * workers, so only the leader apply worker can request walsenders to
+ * delay on the publisher side.
+ */
+ if (server_version >= 160000 && MySubscription->minsenddelay > 0)
+ options.proto.logical.min_send_delay = MySubscription->minsenddelay;
+
/*
* Even when the two_phase mode is requested by the user, it remains
* as the tri-state PENDING until all tablesyncs have reached READY
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 98377c094b..df6a87b0ba 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -285,6 +285,7 @@ parse_output_parameters(List *options, PGOutputData *data)
bool streaming_given = false;
bool two_phase_option_given = false;
bool origin_option_given = false;
+ bool min_send_delay_option_given = false;
data->binary = false;
data->streaming = LOGICALREP_STREAM_OFF;
@@ -396,6 +397,32 @@ parse_output_parameters(List *options, PGOutputData *data)
errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized origin value: \"%s\"", data->origin));
}
+ else if (strcmp(defel->defname, "min_send_delay") == 0)
+ {
+ unsigned long delay_val;
+ char *endptr;
+
+ if (min_send_delay_option_given)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("conflicting or redundant options"));
+ min_send_delay_option_given = true;
+
+ errno = 0;
+ delay_val = strtoul(strVal(defel->arg), &endptr, 10);
+ if (errno != 0 || *endptr != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid min_send_delay")));
+
+ if (delay_val > PG_INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("min_send_delay \"%s\" out of range",
+ strVal(defel->arg))));
+
+ data->min_send_delay = (int32) delay_val;
+ }
else
elog(ERROR, "unrecognized pgoutput option: %s", defel->defname);
}
@@ -501,6 +528,15 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
else
ctx->twophase_opt_given = true;
+ if (data->min_send_delay &&
+ data->protocol_version < LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("requested proto_version=%d does not support delay sending data, need %d or higher",
+ data->protocol_version, LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM)));
+ else
+ ctx->min_send_delay = data->min_send_delay;
+
/* Init publication state. */
data->publications = NIL;
publications_valid = false;
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 2f3c964824..522f7600a1 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -148,7 +148,7 @@ create_logical_replication_slot(char *name, char *plugin,
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* If caller needs us to determine the decoding start point, do so now.
@@ -481,7 +481,7 @@ pg_logical_replication_slot_advance(XLogRecPtr moveto)
XL_ROUTINE(.page_read = read_local_xlog_page,
.segment_open = wal_segment_open,
.segment_close = wal_segment_close),
- NULL, NULL, NULL);
+ NULL, NULL, NULL, NULL);
/*
* Start reading at the slot's restart_lsn, which we know to point to
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 75e8363e24..8cefd8cd0a 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -252,6 +252,7 @@ static void WalSndPrepareWrite(LogicalDecodingContext *ctx, XLogRecPtr lsn, Tran
static void WalSndWriteData(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid, bool last_write);
static void WalSndUpdateProgress(LogicalDecodingContext *ctx, XLogRecPtr lsn, TransactionId xid,
bool skipped_xact);
+static void WalSndDelay(LogicalDecodingContext *ctx, TransactionId xid, TimestampTz delay_start);
static XLogRecPtr WalSndWaitForWal(XLogRecPtr loc);
static void LagTrackerWrite(XLogRecPtr lsn, TimestampTz local_flush_time);
static TimeOffset LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now);
@@ -1126,7 +1127,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
/*
* Signal that we don't need the timeout mechanism. We're just
@@ -1285,7 +1286,7 @@ StartLogicalReplication(StartReplicationCmd *cmd)
.segment_open = WalSndSegmentOpen,
.segment_close = wal_segment_close),
WalSndPrepareWrite, WalSndWriteData,
- WalSndUpdateProgress);
+ WalSndUpdateProgress, WalSndDelay);
xlogreader = logical_decoding_ctx->reader;
WalSndSetState(WALSNDSTATE_CATCHUP);
@@ -3849,3 +3850,75 @@ LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now)
Assert(time != 0);
return now - time;
}
+
+/*
+ * LogicalDecodingContext 'delay' callback.
+ *
+ * Wait long enough to make sure a transaction is applied at least that
+ * period behind the publisher.
+ */
+static void
+WalSndDelay(LogicalDecodingContext *ctx, TransactionId xid, TimestampTz delay_start)
+{
+ /* Wait till delayUntil by the latch mechanism */
+ while (true)
+ {
+ TimestampTz delayUntil;
+ long remaining_wait_time_ms;
+ long timeout_sleeptime_ms;
+
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* This might change wal_sender_timeout */
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+
+ /* Check for input from the client */
+ ProcessRepliesIfAny();
+
+ /* die if timeout was reached */
+ WalSndCheckTimeOut();
+
+ /* Send keepalive if the time has come */
+ WalSndKeepaliveIfNecessary();
+
+ /* Try to flush pending output to the client */
+ if (pq_flush_if_writable() != 0)
+ WalSndShutdown();
+
+ /*
+ * If we've requested to shut down, exit the process.
+ *
+ * Note that WalSndDone() cannot be used here because the delaying
+ * changes will be sent in the function.
+ */
+ if (got_STOPPING)
+ WalSndShutdown();
+
+ delayUntil = TimestampTzPlusMilliseconds(delay_start, ctx->min_send_delay);
+ remaining_wait_time_ms = TimestampDifferenceMilliseconds(GetCurrentTimestamp(), delayUntil);
+
+ /*
+ * Exit without arming the latch if it's already past time to send
+ * this transaction.
+ */
+ if (remaining_wait_time_ms <= 0)
+ break;
+
+ /* Sleep until appropriate time. */
+ timeout_sleeptime_ms = WalSndComputeSleeptime(GetCurrentTimestamp());
+
+ elog(DEBUG2, "time-delayed replication for txid %u, delay_time = %d ms, remaining wait time: %ld ms",
+ xid, (int) ctx->min_send_delay, remaining_wait_time_ms);
+
+ /* Sleep until we get reply from worker or we time out */
+ WalSndWait(WL_SOCKET_READABLE,
+ Min(timeout_sleeptime_ms, remaining_wait_time_ms),
+ WAIT_EVENT_WALSENDER_SEND_DELAY);
+ }
+}
diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c
index cb99cc6339..76c19fe11d 100644
--- a/src/backend/utils/activity/wait_event.c
+++ b/src/backend/utils/activity/wait_event.c
@@ -515,6 +515,9 @@ pgstat_get_wait_timeout(WaitEventTimeout w)
case WAIT_EVENT_VACUUM_TRUNCATE:
event_name = "VacuumTruncate";
break;
+ case WAIT_EVENT_WALSENDER_SEND_DELAY:
+ event_name = "WalSenderSendDelay";
+ break;
/* no default case, so that compiler will warn */
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1a06eeaf6a..9754487921 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4494,6 +4494,7 @@ getSubscriptions(Archive *fout)
int i_subsynccommit;
int i_subpublications;
int i_subbinary;
+ int i_subminsenddelay;
int i,
ntups;
@@ -4546,9 +4547,13 @@ getSubscriptions(Archive *fout)
LOGICALREP_TWOPHASE_STATE_DISABLED);
if (fout->remoteVersion >= 160000)
- appendPQExpBufferStr(query, " s.suborigin\n");
+ appendPQExpBufferStr(query,
+ " s.suborigin,\n"
+ " s.subminsenddelay\n");
else
- appendPQExpBuffer(query, " '%s' AS suborigin\n", LOGICALREP_ORIGIN_ANY);
+ appendPQExpBuffer(query, " '%s' AS suborigin,\n"
+ " 0 AS subminsenddelay\n",
+ LOGICALREP_ORIGIN_ANY);
appendPQExpBufferStr(query,
"FROM pg_subscription s\n"
@@ -4576,6 +4581,7 @@ getSubscriptions(Archive *fout)
i_subtwophasestate = PQfnumber(res, "subtwophasestate");
i_subdisableonerr = PQfnumber(res, "subdisableonerr");
i_suborigin = PQfnumber(res, "suborigin");
+ i_subminsenddelay = PQfnumber(res, "subminsenddelay");
subinfo = pg_malloc(ntups * sizeof(SubscriptionInfo));
@@ -4606,6 +4612,8 @@ getSubscriptions(Archive *fout)
subinfo[i].subdisableonerr =
pg_strdup(PQgetvalue(res, i, i_subdisableonerr));
subinfo[i].suborigin = pg_strdup(PQgetvalue(res, i, i_suborigin));
+ subinfo[i].subminsenddelay =
+ atoi(PQgetvalue(res, i, i_subminsenddelay));
/* Decide whether we want to dump it */
selectDumpableObject(&(subinfo[i].dobj), fout);
@@ -4687,6 +4695,9 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
+ if (subinfo->subminsenddelay > 0)
+ appendPQExpBuffer(query, ", min_send_delay = '%d ms'", subinfo->subminsenddelay);
+
appendPQExpBufferStr(query, ");\n");
if (subinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index cdca0b993d..4c55f8efc4 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -660,6 +660,7 @@ typedef struct _SubscriptionInfo
char *subdisableonerr;
char *suborigin;
char *subsynccommit;
+ int subminsenddelay;
char *subpublications;
} SubscriptionInfo;
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index c8a0bb7b3a..c7d303a168 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6472,7 +6472,7 @@ describeSubscriptions(const char *pattern, bool verbose)
PGresult *res;
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
- false, false, false, false, false, false, false, false};
+ false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6527,10 +6527,13 @@ describeSubscriptions(const char *pattern, bool verbose)
gettext_noop("Two-phase commit"),
gettext_noop("Disable on error"));
+ /* Origin and min_send_delay are only supported in v16 and higher */
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
- ", suborigin AS \"%s\"\n",
- gettext_noop("Origin"));
+ ", suborigin AS \"%s\"\n"
+ ", subminsenddelay AS \"%s\"\n",
+ gettext_noop("Origin"),
+ gettext_noop("Min send delay"));
appendPQExpBuffer(&buf,
", subsynccommit AS \"%s\"\n"
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5e1882eaea..6643db6f55 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1925,7 +1925,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("(", "PUBLICATION");
/* ALTER SUBSCRIPTION <name> SET ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SET", "("))
- COMPLETE_WITH("binary", "disable_on_error", "origin", "slot_name",
+ COMPLETE_WITH("binary", "disable_on_error", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit");
/* ALTER SUBSCRIPTION <name> SKIP ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SKIP", "("))
@@ -3268,7 +3268,7 @@ psql_completion(const char *text, int start, int end)
/* Complete "CREATE SUBSCRIPTION <name> ... WITH ( <opt>" */
else if (HeadMatches("CREATE", "SUBSCRIPTION") && TailMatches("WITH", "("))
COMPLETE_WITH("binary", "connect", "copy_data", "create_slot",
- "disable_on_error", "enabled", "origin", "slot_name",
+ "disable_on_error", "enabled", "min_send_delay", "origin", "slot_name",
"streaming", "synchronous_commit", "two_phase");
/* CREATE TRIGGER --- is allowed inside CREATE SCHEMA, so use TailMatches */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index b0f2a1705d..69ae4314b4 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -74,6 +74,8 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
Oid subowner BKI_LOOKUP(pg_authid); /* Owner of the subscription */
+ int32 subminsenddelay; /* Replication send delay (ms) */
+
bool subenabled; /* True if the subscription is enabled (the
* worker should be running) */
@@ -122,6 +124,7 @@ typedef struct Subscription
* skipped */
char *name; /* Name of the subscription */
Oid owner; /* Oid of the subscription owner */
+ int32 minsenddelay; /* Replication send delay (ms) */
bool enabled; /* Indicates if the subscription is enabled */
bool binary; /* Indicates if the subscription wants data in
* binary format */
diff --git a/src/include/replication/logical.h b/src/include/replication/logical.h
index 5f49554ea0..c389dc17e5 100644
--- a/src/include/replication/logical.h
+++ b/src/include/replication/logical.h
@@ -30,6 +30,11 @@ typedef void (*LogicalOutputPluginWriterUpdateProgress) (struct LogicalDecodingC
bool skipped_xact
);
+typedef void (*LogicalOutputPluginWriterDelay) (struct LogicalDecodingContext *lr,
+ TransactionId xid,
+ TimestampTz start_time
+);
+
typedef struct LogicalDecodingContext
{
/* memory context this is all allocated in */
@@ -64,6 +69,7 @@ typedef struct LogicalDecodingContext
LogicalOutputPluginWriterPrepareWrite prepare_write;
LogicalOutputPluginWriterWrite write;
LogicalOutputPluginWriterUpdateProgress update_progress;
+ LogicalOutputPluginWriterDelay delay_send;
/*
* Output buffer.
@@ -100,6 +106,12 @@ typedef struct LogicalDecodingContext
*/
bool twophase_opt_given;
+ /*
+ * The minimum delay, in milliseconds, by the publisher before sending all
+ * the changes.
+ */
+ int32 min_send_delay;
+
/*
* State for writing output.
*/
@@ -121,14 +133,16 @@ extern LogicalDecodingContext *CreateInitDecodingContext(const char *plugin,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send);
extern LogicalDecodingContext *CreateDecodingContext(XLogRecPtr start_lsn,
List *output_plugin_options,
bool fast_forward,
XLogReaderRoutine *xl_routine,
LogicalOutputPluginWriterPrepareWrite prepare_write,
LogicalOutputPluginWriterWrite do_write,
- LogicalOutputPluginWriterUpdateProgress update_progress);
+ LogicalOutputPluginWriterUpdateProgress update_progress,
+ LogicalOutputPluginWriterDelay delay_send);
extern void DecodingContextFindStartpoint(LogicalDecodingContext *ctx);
extern bool DecodingContextReady(LogicalDecodingContext *ctx);
extern void FreeDecodingContext(LogicalDecodingContext *ctx);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index 0ea2df5088..46faadbd7a 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -36,13 +36,17 @@
* LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM is the minimum protocol version
* where we support applying large streaming transactions in parallel.
* Introduced in PG16.
+ *
+ * LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM is the minimum protocol version
+ * with support for delaying to send transactions. Introduced in PG16.
*/
#define LOGICALREP_PROTO_MIN_VERSION_NUM 1
#define LOGICALREP_PROTO_VERSION_NUM 1
#define LOGICALREP_PROTO_STREAM_VERSION_NUM 2
#define LOGICALREP_PROTO_TWOPHASE_VERSION_NUM 3
#define LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM 4
-#define LOGICALREP_PROTO_MAX_VERSION_NUM LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM
+#define LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM 4
+#define LOGICALREP_PROTO_MAX_VERSION_NUM LOGICALREP_PROTO_MIN_SEND_DELAY_VERSION_NUM
/*
* Logical message types
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index b4a8015403..d2fde09e00 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -30,6 +30,7 @@ typedef struct PGOutputData
bool messages;
bool two_phase;
char *origin;
+ int32 min_send_delay;
} PGOutputData;
#endif /* PGOUTPUT_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index decffe352d..c20969aed7 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -187,6 +187,7 @@ typedef struct
* prepare time */
char *origin; /* Only publish data originating from the
* specified origin */
+ int32 min_send_delay; /* The minimum send delay */
} logical;
} proto;
} WalRcvStreamOptions;
diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h
index 9ab23e1c4a..cc3a234eba 100644
--- a/src/include/utils/wait_event.h
+++ b/src/include/utils/wait_event.h
@@ -150,7 +150,8 @@ typedef enum
WAIT_EVENT_REGISTER_SYNC_REQUEST,
WAIT_EVENT_SPIN_DELAY,
WAIT_EVENT_VACUUM_DELAY,
- WAIT_EVENT_VACUUM_TRUNCATE
+ WAIT_EVENT_VACUUM_TRUNCATE,
+ WAIT_EVENT_WALSENDER_SEND_DELAY
} WaitEventTimeout;
/* ----------
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 3f99b14394..2027316233 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -114,18 +114,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -143,10 +143,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -163,10 +163,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/12345
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/12345
(1 row)
-- ok - with lsn = NONE
@@ -175,10 +175,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
BEGIN;
@@ -210,10 +210,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
ERROR: invalid value for parameter "synchronous_commit": "foobar"
HINT: Available values: local, remote_write, remote_apply, on, off.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+--------------------+------------------------------+----------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | local | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+------------------------------+----------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | 0 | local | dbname=regress_doesnotexist2 | 0/0
(1 row)
-- rename back to keep the rest simple
@@ -247,19 +247,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -271,27 +271,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication already exists
@@ -306,10 +306,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication used more than once
@@ -324,10 +324,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -363,10 +363,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
--fail - alter of two_phase option not supported.
@@ -375,10 +375,10 @@ ERROR: unrecognized subscription parameter: "two_phase"
-- but can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -388,10 +388,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -404,20 +404,61 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 0 | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | 0 | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+ERROR: invalid value for parameter "min_send_delay": "foo"
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+ERROR: -1 ms is outside the valid range for parameter "min_send_delay" (0 .. 2147483647)
+-- fail - specifying streaming = parallel with time-delayed replication is not
+-- supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+ERROR: min_send_delay > 0 and streaming = parallel are mutually exclusive options
+-- success -- min_send_delay value without units is taken as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 123 | off | dbname=regress_doesnotexit | 0/0
+(1 row)
+
+-- success -- min_send_delay value with units other than ms is converted to ms
+-- and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Min send delay | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+----------------+--------------------+----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | 86400000 | off | dbname=regress_doesnotexit | 0/0
(1 row)
+-- fail - alter subscription with streaming = parallel should fail when
+-- time-delayed replication is set
+ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
+ERROR: cannot set parallel streaming mode for subscription with min_send_delay
+-- fail - alter subscription with min_send_delay should fail when
+-- streaming = parallel is set
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 0, streaming = parallel);
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 123);
+ERROR: cannot set min_send_delay for subscription in parallel streaming mode
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 7281f5fee2..46bf4a27d9 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -286,6 +286,35 @@ ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
+-- fail -- min_send_delay must be a non-negative integer
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = foo);
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = -1);
+
+-- fail - specifying streaming = parallel with time-delayed replication is not
+-- supported
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = parallel, min_send_delay = 123);
+
+-- success -- min_send_delay value without units is taken as milliseconds
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexit' PUBLICATION testpub WITH (connect = false, min_send_delay = 123);
+\dRs+
+
+-- success -- min_send_delay value with units other than ms is converted to ms
+-- and stored as an integer
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = '1 d');
+\dRs+
+
+-- fail - alter subscription with streaming = parallel should fail when
+-- time-delayed replication is set
+ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
+
+-- fail - alter subscription with min_send_delay should fail when
+-- streaming = parallel is set
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 0, streaming = parallel);
+ALTER SUBSCRIPTION regress_testsub SET (min_send_delay = 123);
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
RESET SESSION AUTHORIZATION;
DROP ROLE regress_subscription_user;
DROP ROLE regress_subscription_user2;
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 91aa068c95..063a98fde9 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -515,6 +515,33 @@ $node_publisher->poll_query_until('postgres',
or die
"Timed out while waiting for apply to restart after renaming SUBSCRIPTION";
+# Test time-delayed logical replication
+#
+# If the subscription sets min_send_delay parameter, the walsender will delay
+# the transaction send for min_send_delay milliseconds. We verify this by
+# looking at the time difference between a) when tuples are inserted on the
+# publisher, and b) when those changes are replicated on the subscriber. Even
+# on slow machines, this strategy will give predictable behavior.
+
+# Set min_send_delay parameter to 3 seconds
+my $delay = 3;
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub_renamed SET (min_send_delay = '${delay}s')");
+
+# Before doing the insertion, get the current timestamp that will be
+# used as a comparison base.
+my $publisher_insert_time = time();
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_ins VALUES (generate_series(1101, 1120))");
+
+# The publisher waits for the replication to complete
+$node_publisher->wait_for_catchup('tap_sub_renamed');
+
+# This test is successful only if at least the configured delay has elapsed.
+ok( time() - $publisher_insert_time >= $delay,
+ "subscriber applies WAL only after replication delay for non-streaming transaction"
+);
+
# check all the cleanup
$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub_renamed");
--
2.27.0
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-27 06:10 Amit Kapila <[email protected]>
parent: Hayato Kuroda (Fujitsu) <[email protected]>
0 siblings, 1 reply; 17+ messages in thread
From: Amit Kapila @ 2023-02-27 06:10 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: [email protected] <[email protected]>; Peter Smith <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
On Thu, Feb 23, 2023 at 5:40 PM Hayato Kuroda (Fujitsu)
<[email protected]> wrote:
>
> Thank you for reviewing! PSA new version.
>
I was trying to think if there is any better way to implement the
newly added callback (WalSndDelay()) but couldn't find any. For
example, one idea I tried to evaluate is whether we can merge it with
the existing callback WalSndUpdateProgress() or maybe extract the part
other than progress tracking from that function into a new callback
and then try to reuse it here as well. Though there is some common
functionality like checking for timeout and processing replies still
they are different enough that they seem to need separate callbacks.
The prime purpose of a callback for the patch being discussed here is
to delay the xact before sending the commit/prepare whereas the
existing callback (WalSndUpdateProgress()) or what we are discussing
at [1] allows sending the keepalive message in some special cases
where there is no communication between walsender and walreceiver.
Now, the WalSndDelay() also tries to check for timeout and send
keepalive if necessary but there is also dependency on the delay
parameter, so don't think it is a good idea of trying to combine those
functionalities into one API.
Thoughts?
[1] - https://www.postgresql.org/message-id/20230210210423.r26ndnfmuifie4f6%40awork3.anarazel.de
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: Time delayed LR (WAS Re: logical replication restrictions)
@ 2023-02-27 07:04 Hayato Kuroda (Fujitsu) <[email protected]>
parent: Amit Kapila <[email protected]>
0 siblings, 0 replies; 17+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2023-02-27 07:04 UTC (permalink / raw)
To: 'Amit Kapila' <[email protected]>; +Cc: [email protected] <[email protected]>; Peter Smith <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Takamichi Osumi (Fujitsu) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; pgsql-hackers
Dear Amit,
> I was trying to think if there is any better way to implement the
> newly added callback (WalSndDelay()) but couldn't find any. For
> example, one idea I tried to evaluate is whether we can merge it with
> the existing callback WalSndUpdateProgress() or maybe extract the part
> other than progress tracking from that function into a new callback
> and then try to reuse it here as well. Though there is some common
> functionality like checking for timeout and processing replies still
> they are different enough that they seem to need separate callbacks.
> The prime purpose of a callback for the patch being discussed here is
> to delay the xact before sending the commit/prepare whereas the
> existing callback (WalSndUpdateProgress()) or what we are discussing
> at [1] allows sending the keepalive message in some special cases
> where there is no communication between walsender and walreceiver.
> Now, the WalSndDelay() also tries to check for timeout and send
> keepalive if necessary but there is also dependency on the delay
> parameter, so don't think it is a good idea of trying to combine those
> functionalities into one API.
>
> Thoughts?
>
> [1] -
> https://www.postgresql.org/message-id/20230210210423.r26ndnfmuifie4f6%40
> awork3.anarazel.de
Thank you for confirming. My understanding was that we should keep the current design.
I agree with your posting.
In the current callback and modified version in [1], sending keepalives is done
via ProcessPendingWrites(). It is called by many functions and should not be changed,
like adding end_time only for us. Moreover, the name is not suitable because
time-delayed logical replication does not wait until the send buffer becomes empty.
If we reconstruct WalSndUpdateProgress() and change mechanisms around that,
codes will become dirty. As Amit said, in one path, the lag will be tracked and
the walsender will wait until the buffer is empty.
In another path, the lag calculation will be ignored, and the walsender will wait
until the process spends time till a given period. Such a function is painful to read later.
I think callbacks that have different purposes should not be mixed.
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 17+ messages in thread
end of thread, other threads:[~2023-02-27 07:04 UTC | newest]
Thread overview: 17+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-12-16 20:24 [PATCH 6/8] BRIN bloom indexes Tomas Vondra <[email protected]>
2023-02-16 08:55 Re: Time delayed LR (WAS Re: logical replication restrictions) Kyotaro Horiguchi <[email protected]>
2023-02-17 06:44 ` RE: Time delayed LR (WAS Re: logical replication restrictions) Hayato Kuroda (Fujitsu) <[email protected]>
2023-02-18 05:48 ` Re: Time delayed LR (WAS Re: logical replication restrictions) Amit Kapila <[email protected]>
2023-02-20 02:27 ` RE: Time delayed LR (WAS Re: logical replication restrictions) Hayato Kuroda (Fujitsu) <[email protected]>
2023-02-20 22:01 ` Re: Time delayed LR (WAS Re: logical replication restrictions) Peter Smith <[email protected]>
2023-02-21 02:00 ` Re: Time delayed LR (WAS Re: logical replication restrictions) Amit Kapila <[email protected]>
2023-02-21 08:03 ` RE: Time delayed LR (WAS Re: logical replication restrictions) Hayato Kuroda (Fujitsu) <[email protected]>
2023-02-21 07:57 ` RE: Time delayed LR (WAS Re: logical replication restrictions) Hayato Kuroda (Fujitsu) <[email protected]>
2023-02-22 11:55 ` Re: Time delayed LR (WAS Re: logical replication restrictions) Amit Kapila <[email protected]>
2023-02-22 13:47 ` RE: Time delayed LR (WAS Re: logical replication restrictions) Hayato Kuroda (Fujitsu) <[email protected]>
2023-02-23 01:19 ` Re: Time delayed LR (WAS Re: logical replication restrictions) Peter Smith <[email protected]>
2023-02-23 08:55 ` RE: Time delayed LR (WAS Re: logical replication restrictions) [email protected] <[email protected]>
2023-02-23 12:09 ` RE: Time delayed LR (WAS Re: logical replication restrictions) Hayato Kuroda (Fujitsu) <[email protected]>
2023-02-27 06:10 ` Re: Time delayed LR (WAS Re: logical replication restrictions) Amit Kapila <[email protected]>
2023-02-27 07:04 ` RE: Time delayed LR (WAS Re: logical replication restrictions) Hayato Kuroda (Fujitsu) <[email protected]>
2023-02-21 01:06 ` Re: Time delayed LR (WAS Re: logical replication restrictions) Peter Smith <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox